Functions and Restrictions of the asys Tool

Functions

To improve the system fault maintenance and test efficiency, the fault information collection tool asys is provided to collect information in one-click mode. : This tool can be used only in Ascend EP mode.

The tool supports the following functions:

Fault information collection: collects fault information, such as software and hardware information and logs, without stopping services.
Service restoration + fault information collection: Collect fault information after services are restored, such as software and hardware information and logs.
Displays software, hardware, and device status information: Collects the installation package version information, device temperature, and power.
Health Check: Checks the health status of all devices or a specified device. If the device is unhealthy, an error message is displayed.
Comprehensive detection: involves functions such as stress detection, HBM hardware detection, and CPU detection.
Component detection: Currently, only AI Vector component detection is supported. Parallel execution is not supported.
trace file parsing /coredump file parsing /stackcore file parsing /coretrace file parsing /UB file parsing: parses various files to facilitate subsequent fault locating.
Real-time stack export: This function applies to scenarios where service processes are suspended to export stack information for fault locating.
Environment Configuration: Obtain or restore specified configurations.
AI Core error information parsing: During service execution, if the log file or screen printing information contains AI Core errors (for example, "there is an aivec error exception" or" there is an aicore error exception"), you can use this function to quickly locate the causes of the AI Core errors.
Performance Data Collection: collects key performance data to help users analyze performance problems.

**Table 1** Information that can be collected by the asys tool
Category	Description
Software information	Software package version, environment variables, software dependency, and system information.
Log information	The information includes: CANN software stack logs on the host. Message logs on the host. Device firmware logs: device-* logs (requiring the root permission) Device system logs: message logs and device-os logs (requiring the root permission) Black Box, stackcore, and coretrace files (requiring the root permission). Task print log. Runfile installation logs (available only when the runfile installation user is the same as the application execution user)
Dump information	The information includes: Dumped GE graphs. Dumped TF Adapter graphs. Dump file generated when an AI Core error occurs.
.o and .json files for operator compilation	-
Operator compilation process file	Only the operator compilation process information is collected during service re-run. The information includes compilation success or failure, reused memory, online compilation, and binary compilation results. Whether the asys tool can collect the operator compilation process information depends on whether the NPU_COLLECT_PATH environment variable (used to set the path for saving fault information) is specified. If it is set, the system creates the /extra-info/ops/ subdirectory in the directory specified by the environment variable, creates op_compile_stats.log in the subdirectory, and writes the operator compilation process information to the log file. In this case, the asys tool can collect the operator compilation process information. If this environment variable is not set, the system does not generate the corresponding log file. Therefore, the asys tool does not collect the file.
Custom operator configuration (*.json file)	Whether the asys tool can collect the custom operator configuration depends on whether the following environment variables are set: If the ASCEND_OPP_PATH environment variable (used to set the installation path of the operator library) is set, the asys tool collects the custom operator configuration (that is, the *config/.json** file) in the ${ASCEND_OPP_PATH}/vendors directory based on the load_priority field in the ${ASCEND_OPP_PATH}/vendors/config.ini file. Otherwise, the asys tool does not collect the information. If the ASCEND_CUSTOM_OPP_PATH environment variable (used to set the installation path of the custom operator package) is set, the custom operator configuration (that is, the *config/.json file) in the ${ASCEND_CUSTOM_OPP_PATH}** directory is collected. Otherwise, the asys tool does not collect the information.
Commands executed in user cases	-
Binary information of the debugging version	Information in the ${ASCEND_OPP_PATH}/debug_kernel directory. You need to configure the ASCEND_OPP_PATH environment variable (used to set the installation directory of the operator library) in advance. If the ASCEND_OPP_PATH environment variable is not configured or incorrectly configured, the binary information of the debugging version is not collected by default.

For details about how to set environment variables, see Environment Variables.

Restrictions

This function cannot be used in Ascend RC mode.
If more than one process is operated by the same user on a machine at the same time, the collected data may overlap.
Only limited data can be collected by a non-root user. For details about the limitations, see the privilege requirements in Functions.
The one-click tool cannot be used to collect fault information in cluster, container, VM, and cloud scenarios.
The asys tool collects a large amount of maintenance and test information. Therefore, memory usage is involved. You are advised not to run multiple processes in parallel. Otherwise, an error may occur during the execution of the asys tool or the environment may encounter exceptions.
The asys tool searches for the related information in the directory where trace logs are stored. If there are too many trace log files, the execution of the asys tool may take a long time.
The default path for storing trace logs is $HOME/ascend/atrace/. For details about trace logs, see Viewing Trace Logs in Log Reference.

Parent topic: asys Tool Usage Guide