Overview

msDebug is a tool used to debug operator programs running on the NPU. It specially empowers operator developers to debug operators on Ascend devices. The debugging methods include reading the memory and register of an Ascend device, and pausing and resuming the running status of a program. After testing the operator functions in a real-world hardware environment by starting operators or using the msOpST tool, you can determine whether to use the msDebug tool for function debugging based on the actual test situation.

  • To enable msDebug, specify the --debug parameter when installing the driver. For details about how to install the driver, see Installing the NPU Driver and Firmware.
    ./Ascend-hdk-<chip_type>-npu-driver_<version>_linux-<arch>.run --debug
  • The debugging channel has high permissions, which causes security risks. Exercise caution when using this tool. This tool is not recommended in the production environment. If you use this tool, you implicitly accept the risks involved.

Features

The msDebug tool can debug all Ascend operators, including Ascend C operators (Vector, Cube, and mix fused operators). You can select an Ascend operator as needed. For details, see Table 1.

Table 1 msDebug functions

Function

Link

Breakpoint settings

Setting Breakpoints

Variable and memory printing

Printing Memory and Variables

Step-by-step debugging

Executing Step-by-Step Debugging

Running interruption

Interrupting Execution

Core switchover

Switching Cores

Program status check

Reading Register Values

Debugging information display

Displaying the Debugging Information

Commands

  • You need to ensure the execution security of executable files or applications.
    • You are advised to restrict the operation permission on executable files or applications to avoid privilege escalation risks.
    • You are not advised to perform high-risk operations (such as deleting files, deleting directories, changing passwords, and running privilege escalation commands) to avoid security risks.
  • You can run the help command to view all the commands supported by msDebug. Commands excluded in Table 2 are implemented by the open-source debugger LLDB. Pay attention to related risks when using LLDB. For details about how to use LLDB, see its official document at https://lldb.llvm.org/.
Table 2 Command reference

Command

Abbreviation

Description

Example

breakpoint set -f filename -l linenum

b

Adds a breakpoint. filename is the operator implementation code file *.cpp, and linenum is the line number of the code file.

b add_custom.cpp:85

run

r

Runs a program.

r

continue

c

Keeps running.

c

print variable

p

Prints variables.

p zLocal

frame variable

var

Displays all local variables in the current scope.

var

memory read

x

Reads the memory.

x -m GM -f float16[] 0x00001240c0037000 -c 2 -s 128
  • -m specifies the memory location. GM, UB, L0A, L0B, L0C, L1, and FB are supported.
  • -s specifies the number of bytes to be printed in each line.
  • -c specifies the number of lines to be printed.
  • -f specifies the type of the data to be printed.
  • 0x00001240c0037000 indicates the memory address to be read. Change it based on the actual situation.

ascend info devices

-

Queries device information.

ascend info devices

ascend info cores

-

Queries information about the AI Core running on an operator.

ascend info cores

ascend info tasks

-

Queries information about the task running on an operator.

ascend info tasks

ascend info stream

-

Queries information about the stream running on an operator.

ascend info stream

ascend info blocks

-

Queries information about the block running on an operator.

To display information about running blocks:
ascend info blocks 
To display the code of the running blocks at the current breakpoint:
ascend info blocks -d

ascend aic id

-

Switches the target cube core of the debugger.

ascend aic 1

ascend aiv id

-

Switches the target vector core of the debugger.

ascend aiv 5

CTRL+C

-

Manually interrupts the operator execution program and displays the interrupted location information.

Enter information using the keyboard.

register read

re r

Reads register values. -a reads all register values. $REG_NAME reads the value of a specified register.

register read -a
re r $PC

thread step-over

next or n

Moves to the next executable line of code in the same call stack.

n

thread step-in

step or s

Switches to the internal function for debugging.

s

thread step-out

finish

Executes the remaining part of the function and returns to the main program.

finish

target modules add <kernel.o>

image add [kernel.o]

Imports operator debugging information when the PyTorch framework calls operators.

NOTE:

After the run command is executed, run the image add command to import the debugging information. Then, run the image load command for the imported debugging information to take effect.

image add xx.o       

target modules load --file <kernel.o> --slide <address>

image load -f <kernel.o> -s <address>

Loads the operator debugging information for the imported debugging information to take effect when the PyTorch framework calls an operator.

image load -f xx.o -s 0

help msdebug_command

-

Displays the help information.

NOTE:

The command output displays the function, syntax, and options of a command.

Help information for core switchover:
1
2
3
(msdebug) help ascend aic
change the id of the focused ascend aicore.
Syntax: ascend aic <id>

Help information for ascend info blocks:

1
2
3
4
5
6
7
(msdebug) help ascend info blocks
show blocks overall info.
Syntax: ascend info blocks
Command Options Usage:
  ascend info blocks [-d]
       -d ( --details )
            Show stopped states for all blocks.
help run

Call Scenarios

The following operator call scenarios are supported:

Supplementary Notes

msDebug also provides the following extension program. For details, see Table 3.

Table 3 Description of the extension program

Program Name

Program Description

Description

msdebug-mi

This program is used together with the MindStudio-Debug-VSCode-Plugin plugin to implement visualized debugging.

Machine-machine interface, which can be ignored by users.