Collect Information About Process Interruption
- Manual collection: Collect host and device logs and files. Only the minimum set of information is collected.
- Automatic collection: On the host server, use the asys tool to collect all fault-related information, including the installation version information, device health status information, dump files, operator compilation information, and full log files.
Note: The asys tool can be used to collect fault information in only limited scenarios, excluding cluster, container, virtual machines, and cloud.
Manual Collection
- Plan a directory for storing log files on the host server, for example, ${HOME}/err_log_info/.
- The default path of application log files is ${HOME}/ascend/log on the host server. Move the log files to the err_log_info directory.
mv ${HOME}/ascend/log ${HOME}/err_log_info/ - Use the msnpureport tool to export device-side system logs and other maintenance and test information to the host, including slogs, syslogs, and black box logs.
# Create a directory for storing logs and files in the ${HOME}/err_log_info directory. cd ${HOME}/err_log_info mkdir report # Run the msnpureport command in the report directory. cd report msnpureport -f
For details about log levels, log paths, and log files, see Log Reference.
In addition, when locating problems, technical support may need onsite service information and user operation logs. Onsite service information indicates whether the onsite service is a single operator, model inference, or model training service. If the onsite service is a training service, the onsite service information contains scale of the training cluster and etc. User operation logs record user operations on the host server. Based on this part, technical support can learn about the basic onsite information and check whether the process is manually interrupted.
Automatic Collection
For details about the restrictions on using the asys tool, see Functions and Restrictions of the asys Tool. Before using the asys tool, install and configure it. For details, see the prerequisites in Preparing the asys Tool Environment.
The following is an example of the asys tool command. Run the asys collect command to collect fault information.
asys collect --output="path"
output indicates the directory where the collected information is stored. For details about the parameters and restrictions, see Fault Information Collection.