Tool Usage

msprof op

1. Log in to the operating environment and run the msprof op optional parameter app [arguments] to enable on-board operator tuning. For details about the optional parameters, see Table 2. An example command is as follows:

   msprof op --output=$HOME/projects/output $HOME/projects/MyApp/out/main  // --output is optional. $HOME/projects/MyApp/out/main is the application in use.

2. Perform operator tuning in either of the following ways:
   • Executable file-based method
     • Single-operator scenario (using add_custom_npu as an example)
       Example 1:

       msprof op ./add_custom_npu

       Example 2:

       msprof op --aic-metrics=<select_metrics> --output=./output_data ./add_custom_npu 

     • Multi-operator scenario
       If the test executable contains Add, MatlMul, and Sub operators, you can use --launch-count and --kernel-name to specify collecting data for the Add and Sub operators.

       msprof op --launch-count=10 --kernel-name="Add|Sub" --output=./output_data ./test  // ./test is the user binary file and should be placed at the end of the command.

   • Method based on the JSON configuration file of the input operator binary file *.o. For details, see JSON Configuration File Description.

     msprof op --config=./add_test.json --aic-metrics=<select_metrics> --output=./output_data

3. After the command is executed, a folder named OPPROF_{timestamp}_XXX is generated in the default path or the specified --output directory. When all --aic-metrics are enabled, the structure is as follows:
   • Collecting data in the multi-device multi-operator scenario.
     

     When tuning MC2 or LCCL fused operators in multi-device parallel mode, several subdirectories named after device IDs will exist in the result directory, depending on the specified number of NPUs. The tuning results of each NPU are stored in the corresponding device ID directory.

     1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19

     └──OPPROF_{timestamp}_XXX ├── device0 // ID of the Ascend AI Processor used during running └── device1 ├── OpName0 // OpName0 is the name of the collection operator. │ ├── 0 // Sequence in which operators are scheduled. │ │ ├──dump // Folder for storing the process files. The meaning of this parameter is the same as that in single-operator collection. │ │ └──xxx_yyy.csv // xxx indicates the type of the metric generated by an operator, for example, L2Cache. For details about the metric types, see the description of the csv. file in . yyy indicates the timestamp suffix of the .csv file, for example, L2Cache_20240603022812284.csv. │ │ └──visualize_data.bin ├── OpName1 │ ├── 0 │ │ ├──dump │ │ └──xxx_yyy.csv │ │ └──visualize_data.bin ├── OpName2 │ ├── 0 │ │ ├── dump │ │ └── xxx_yyy.csv │ │ └──visualize_data.bin │ │ └── trace.json // Applicable only to MC2 and LCCL operators.

   • Collecting data of multiple operators on a single device

     1 2 3 4 5 6 7 8 9 10 11 12 13 14 15

     └──OPPROF_{timestamp}_XXX ├── OpName0 // OpName0 is the name of the collection operator. │ ├── 0 // Sequence in which operators are scheduled. │ │ ├── dump // Folder for storing the process files. The meaning of this parameter is the same as that in single-operator collection. │ │ └── xxx_yyy.csv // xxx indicates the type of the metric generated by an operator, for example, L2Cache. For details about the metric types, see the description of the csv. file in . yyy indicates the timestamp suffix of the .csv file, for example, L2Cache_20240603022812284.csv. │ │ └──visualize_data.bin │ ├── 1 │ │ ├──dump │ │ └──xxx_yyy.csv │ │ └──visualize_data.bin ├── OpName1 │ ├── 0 │ │ ├── dump │ │ └── xxx_yyy.csv │ │ └── visualize_data.bin

   • Collecting data of a single-operator on a single device

     1 2 3 4 5 6 7 8 9 10 11

     OPPROF_{timestamp}_XXX ├── dump ├── ArithmeticUtilization.csv ├── L2Cache.csv ├── Memory.csv ├── MemoryL0.csv ├── MemoryUB.csv ├── OpBasicInfo.csv ├── PipeUtilization.csv ├── ResourceConflictRatio.csv ├── visualize_data.bin

   Table 2 msprof op files

   File	Description
   dump folder	Raw profile data, which can be ignored.
   ArithmeticUtilization.csv	Time consumptions and ratios of Cube and Vector instructions. For details, see ArithmeticUtilization (Time Consumptions and Percentages of Cube and Vector Instructions).
   L2Cache.csv	L2 cache hit ratio. For details, see L2Cache (L2 Cache Hit Ratio).
   Memory.csv	UB/L1/L2/main memory read/write bandwidth rate. For details, see Memory (Memory Read/Write Bandwidth Rate).
   MemoryL0.csv	L0A/L0B/L0C memory read/write bandwidth rate. For details, see MemoryL0 (L0 Read/Write Bandwidth Rate).
   MemoryUB.csv	MTE/Vector/Scalar UB read/write bandwidth rate. For details, see MemoryUB (UB Read/Write Bandwidth Rate).
   PipeUtilization.csv	Time consumptions and ratios of compute units and MTE units. For details, see PipeUtilization (Percentages of Time Taken by Compute Units and MTEs).
   ResourceConflictRatio.csv	Ratios of bank groups, bank conflicts, and resource conflicts in the UB to all instructions. For details, see ResourceConflictRatio (Resource Conflict Ratio).
   OpBasicInfo.csv	Basic operator information, including the operator names, block dim, and time consumptions. For details, see OpBasicInfo (Basic Operator Information).
   visualize_data.bin	File that displays basic operator information, compute unit load, hotspot functions, and Roofline bottleneck analysis. For details, see Computing Memory Heatmap, Roofline Bottleneck Analysis Chart, Cache Heatmap, Communication and Computing Pipeline Chart, and Operator Code Hot Spot Map. NOTE: visualize_data.bin can be displayed in a visual format on MindStudio Insight. For details, see MindStudio Insight User Guide. The msprof op hotspot function is supported only by the Atlas A2 training products/Atlas A2 inference products. Currently, only the Communication and Computing Pipeline Chart can be generated for MC2 and LCCL fused operators. Cache Heatmap and Operator Code Hot Spot Map cannot be generated for MC2 and LCCL fused operators or for Atlas inference products. The unit GB/s indicates that 1 GB of data is transmitted per second.
   trace.json	File for visualizing the communication and computing pipelines. For details about how to use the Chrome browser to display the file, see Communication and Computing Pipeline Chart.

4. After the visualize_data.bin file is imported to the MindStudio Insight, Computing Memory Heatmap, Roofline Bottleneck Analysis Chart, Cache Heatmap, Communication and Computing Pipeline Chart, and Operator Code Hot Spot Map are displayed.
5. After the trace.json file is imported to the Chrome browser or MindStudio Insight, Communication and Computing Pipeline Chart is displayed.

msprof op simulator

The operator tuning tool supports profile data collection and automatic parsing in a simulation environment.

• The collection of MC2 and HCCL operators is not supported in the simulation environment.
• The number of simulation cores set by the user cannot exceed the number of physical cores.
• For performance of some operators, call TRACE_START and TRACE_STOP APIs in the single core of Atlas A3 training products/Atlas A3 inference products, Atlas inference products, and Atlas A2 training products/Atlas A2 inference products. Add -DASCENDC_TRACE_ON to the compilation configuration file. For details, see adding -DASCENDC_TRACE_ON. Then, the system can generate the pipeline chart. For details about the flow chart content, see Instruction Pipeline Chart.
• You need to add -DASCENDC_TRACE_ON to the compilation configuration file. For details, see the following sample project.
  For AddKernelInvocationNeo operator project, add the following code to the ${git_clone_path}/samples/operator/ascendc/0_introduction/3_add_kernellaunch/AddKernelInvocationNeo/cmake/npu_lib.cmake file:

  ascendc_compile_definitions
  (
      ...
      -DASCENDC_TRACE_ON
  )