Debugging on the CPU

This section describes how to debug operators on the CPU, including GDB debugging and printing using the printf command.

The debugging methods provided in this section are based on the operator program in Kernel Launch. Ensure that you have learnt the runtime verification of the kernel function in related sections.

During CPU debugging, configuring log-related environment variables can help you record program running processes and exceptions, and debug functions.

For details about the restrictions and description of environment variables, see Logs.

GDB Debugging

You can use GDB to debug the operator computation precision step by step. The CPU debugging has been changed to multi-process debugging, and each core starts over an independent subprocess. Therefore, the GDB needs to be changed to the subprocess debugging mode. For the Atlas Training Series Product , each core starts one subprocess.

  • Debug a single subprocess.
    After the GDB is started, run the following command to set the tracing subprocess and then add breakpoints to the subprocess:
    set follow-fork-mode child

    However, this method stays only in the first subprocess that encounters a breakpoint, and other subprocesses and the main process continue to execute until they exit. Operators involving inter-core synchronization cannot be debugged using this method.

  • Debug multiple subprocesses.

    If inter-core synchronization is involved, multiple subprocesses need to be debugged in parallel.

    After the GDB is started, set the debugging mode to debug only one process and suspend other processes. The command is as follows:

    1
    (gdb) set detach-on-fork off
    

    Run the following command to view the current debugging mode:

    1
    (gdb) show detach-on-fork
    

    The GDB program captures and interrupts the fork event. In this way, the GDB program can be interrupted each time a subprocess is started. The command is as follows:

    1
    (gdb) catch fork
    

    After r is executed, you can view the current process information.

    1
    2
    3
    (gdb) info inferiors
      Num  Description
    * 1    process 19613
    

    When the fork command is executed for the first time, the program is disconnected from the fork position of the main process, and the subprocess is not generated.

    After c is executed, check info inferiors again. You can see that the first subprocess is started.

    1
    2
    3
    4
    (gdb) info inferiors
      Num  Description 
    * 1    process 19613
      2    process 19626
    

    In this case, you can switch to the second process, that is, the first subprocess, and then add a breakpoint for debugging. In this case, the main process is suspended.

    1
    2
    3
    4
    5
    6
    (gdb) inferior 2
    [Switching to inferior 2 [process 19626] ($HOME/demo)]
    (gdb) info inferiors
      Num  Description
      1    process 19613
    * 2    process 19626
    

    Note that the number following inferior is the sequence number of the process, not the process number.

    If synchronization is blocked, you can switch back to the main process to continue generating subprocesses, and then switch to a new subprocess for debugging. After the synchronization conditions are met, switch back to the first subprocess to continue execution.

The following is an example of commands for debugging a single subprocess:

gdb --args add_custom_cpu
set follow-fork-mode child
break add_custom.cpp:45
run
list
backtrace
print i
break add_custom.cpp:56
continue
display xLocal
quit

printf

Compile printf(...) in the code to observe the value output. The sample code is as follows:
printf("xLocal size: %d\n", xLocal.GetSize()); 
printf("tileLength: %d\n", tileLength);