Performance Tuning

Performance Collection and Analysis Tool

After an executable program is generated using the BiSheng compiler, you can use the msProf tool to run the executable file generated in NPU mode to collect the performance data of the Ascend C operator executed on the AI processor for refined performance tuning.
  • Profiling performance data collection: Use the msProf tool to collect the performance data of the Ascend C operator executed on the AI processor.
  • Roofline bottleneck analysis: The visualize_data.bin file generated by msprof op can be visualized using MindStudio Insight. A Roofline bottleneck analysis chart can be used to build a processor performance model, which can be used to quickly evaluate the theoretical performance limit of an operator, allowing you to quickly identify bottlenecks.
  • Instruction pipeline chart analysis: The visualize_data.bin or trace.json file generated by msprof op simulator can be used for visualized display. An instruction pipeline chart displays timing relationship by instruction and associates with the call stack to quickly locate bottlenecks.

For details about how to use the performance tuning tool, see Operator Tuning (msProf).

Board Performance Tuning on the NPU

After an operator program is compiled into an executable program using the BiSheng compiler, you can use msprof op to collect performance data on the NPU. Take the SIMD programming scenario as an example. The procedure of using msProf to collect board performance data is as follows:

  1. Compile the add operator sample to generate an executable file by referring to AI Core SIMD Programming.

    The number following dav- indicates the NPU architecture version. Replace it with the actual one. For details about the architecture version number of each product model, see Table 1.

    1
    bisheng add_custom.asc -o add_custom --npu-arch=dav-2201

  2. Use msprof op to call the operator executable file for performance data collection.

    1
    msprof op ./add_custom

  3. View the performance data to learn about the performance bottlenecks of the current operator.

     1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
    The following is an example of the profile data folder structure:
├──dump                       # Raw performance data, which you can ignore.
├──ArithmeticUtilization.csv  # Cube/Vector instruction cycle ratio. You are advised to optimize operator logic and reduce redundant computation instructions.
├──L2Cache.csv                # L2 cache hit rate, which affects MTE2. You are advised to properly plan the data transfer logic to increase the hit rate.
├──Memory.csv                 # UB, L1, and main memory read/write bandwidth (GB/s).
├──MemoryL0.csv               # L0A, L0B, and L0C read/write bandwidth (GB/s).
├──MemoryUB.csv               # Vector and scalar read/write bandwidth to/from the UB (GB/s).
├──OpBasicInfo.csv            # Basic operator information.
├──PipeUtilization.csv        # Pipe class instruction execution time and ratio. It is advised to optimize data movement logic to improve bandwidth utilization.
├──ResourceConflictRatio.csv  # Ratio of bank groups, bank conflicts, and resource conflicts in the UB to all instructions. You are advised to reduce or avoid read/write conflicts on the same bank or read/read conflicts on a bank group.
└──visualize_data.bin         # File for presentation in MindStudio Insight.

In the SIMT programming scenario, you only need to follow the AI Core SIMT compilation guide to compile the operator. After the executable file is generated, use the msProf tool to execute the program by following steps 2 and 3 above to obtain the operator execution performance data.

NPU Performance Simulation

On non-Ascend devices, after using the BiSheng compiler for simulation compilation to generate an executable program, you can use msprof op simulator to complete performance pipeline simulation. Currently, only the SIMD programming scenario is supported. The SIMT programming scenario is not supported.

Parent topic: Debugging and Tuning