Before You Start

Environment Setup

Configure environment variables by referring to Environment Setup.

Restrictions

• To enable the msDebug tool, you need to install the driver. For details, see Overview.
• For a single device, only one msDebug tool can be used for debugging. You are not advised to run other operator programs at the same time.
• When the program to be debugged calls multiple operators, the msDebug tool can debug only a specified operator.
• During operator debugging, the overflow/underflow detection function is disabled.

Importing Debugging Information

Before debugging an operator, enable the -g -O0 option and recompile the operator so that the operator binary contains the debugging information. For details, see compile the operator based on the sample project.

The behavior of the operator program in the -O0 compilation option scenario is different from that in the -O2 compilation scenario. Therefore, you are advised to use the Contention Check function of the msSanitizer tool instead of the -O0 compilation option to locate the contention issue in an operator.

Generally, the operator debugging information is automatically imported to the msDebug tool. However, if the .o operator binary file exists independently and is deployed (for example, in the AscendCL single-operator calling scenario), import the operator debugging information as follows.

• In the multi-operator scenario, only the debugging information of a specified single operator can be imported. Only the .o file of the specified single operator can be debugged.
• Multiple .o files are generated during complex operator compilation. For details about how to select a specific .o file, see Ambiguous .o File Selection When LAUNCH_KERNEL_PATH Is Configured to Import Operator Debugging Information.

• Method 1: Before debugging, configure the following environment variable, specify the operator loading path, and import the debugging information.

  export LAUNCH_KERNEL_PATH={path_to_kernel} /my_kernel.o // {path_to_kernel} is the directory where the .o file of the kernel is stored.

• Method 2: Before executing the run command, run the image add command to specify the operator loading path and import the debugging information.

  (msdebug) image add {path_to_kernel} /my_kernel.o // {path_to_kernel} is the directory where the .o file of the kernel is stored.

  

  • image add applies only to the PyTorch scenario.
  • If you want to import the debugging information after the program runs, run the image load command to load the operator debugging information.

    (msdebug) image load -f {path_to_kernel}/my_kernel.o -s 0 

Starting the Tool

The msDebug tool can be started in either of the following ways.

If Cannot read termcap database; using dumb terminal settings. is displayed, run the export TERMINFO=xx command to clear the prompt. xx indicates the local TERMINFO path.

export TERMINFO=xx    //xx can be queried by running the infocmp -D command. You can select a path that meets the current terminal configuration as the value of TERMINFO.

• Load the executable file application.
  1. After the operator is built, the executable file application on the NPU can be obtained.
     

     Perform one-click compilation and running based on the kernel framework of the Ascend C operator to generate the executable file application on the NPU. For details, see "Kernel Launch".

  2. Use msDebug to load the executable file.

     $ msdebug ./application

     

     If the executable file has other input parameters, pass them as follows:

     msdebug -- ./application --flag1 arg1 --flag2 args2 ...

• Load the Python script for operator calling.
  1. After plugins of the PyTorch framework are developed, you can directly call Ascend C custom operators from PyTorch through the custom Python script test_ops_custom.py.
     

     For details about single-operator calling in the PyTorch framework, see Adapting OpPlugin to a Single Operator.

  2. Use msDebug to load the Python script.

     1 2 3 4 5 6 7 8

     $ msdebug python3 test_ops_custom.py msdebug(MindStudio Debugger) is part of MindStudio Operator-dev Tools. The tool provides developers with a mechanism for debugging Ascend kernels running on actual hardware. This enables developers to debug Ascend kernels without being affected by potential changes brought by simulation and emulation environments. (msdebug) target create "python3" Current executable set to '${INSTALL_DIR}/projects/application' (aarch64). (msdebug) settings set -- target.run-args "test_ops_custom.py" (msdebug)

Exiting Debugging

(msdebug) q
[localhost add_ascendc_sample]$ 

The debugging channel cannot be disabled independently. To disable the debugging channel, you need to enable the overwrite mode. For details, see the NPU driver and firmware installation documents.