Collect Information About Process Interruption

The information can be collected manually or automatically.
  • Manual collection: Collect host and device logs and files. Only the minimum set of information is collected.
  • Automatic collection: On the host server, use the asys tool to collect all fault-related information, including the installation version information, device health status information, dump files, operator compilation information, and full log files.

    Note: The asys tool can be used to collect fault information in only limited scenarios, excluding cluster, container, virtual machines, and cloud.

Manual Collection

To collect application logs, device-side system logs, and other maintenance and test information, perform the following steps:
  1. Plan a directory for storing log files on the host server, for example, ${HOME}/err_log_info/.
  2. The default path of application log files is ${HOME}/ascend/log on the host server. Move the log files to the err_log_info directory.
    mv ${HOME}/ascend/log ${HOME}/err_log_info/ 
  3. Use the msnpureport tool to export device-side system logs and other maintenance and test information to the host, including slogs, syslogs, and black box logs.
    # Create a directory for storing logs and files in the ${HOME}/err_log_info directory.
    cd ${HOME}/err_log_info
    mkdir report
    
    # Run the msnpureport command in the report directory.
    cd report
    msnpureport -f

For details about log levels, log paths, and log files, see Log Reference.

In addition, when locating problems, technical support may need onsite service information and user operation logs. Onsite service information indicates whether the onsite service is a single operator, model inference, or model training service. If the onsite service is a training service, the onsite service information contains scale of the training cluster and etc. User operation logs record user operations on the host server. Based on this part, technical support can learn about the basic onsite information and check whether the process is manually interrupted.

Automatic Collection

For details about the restrictions on using the asys tool, see Functions and Restrictions of the asys Tool. Before using the asys tool, install and configure it. For details, see the prerequisites in Preparing the asys Tool Environment.

The following is an example of the asys tool command. Run the asys collect command to collect fault information.

asys collect --output="path"

output indicates the directory where the collected information is stored. For details about the parameters and restrictions, see Fault Information Collection.