Documentation
Download

Profiling AI Tasks When They Are Running

Function

msprof supports profiling of AI tasks when they are running. After the profiling, it automatically parses the profile data and flush corresponding files to disks.

Command Format

For Ascend EP, log in to the operating environment and run the command in any directory.
msprof [options] <app>
For Ascend RC, log in to the operating environment, go to the /var directory where msprof is installed, and run the command.
./msprof [options] <app>

app is mandatory. For details about the parameters, see app Parameters. For details about the options, see Command-line Options.

Command-line Options

  • --ascendcl=<ascendcl-value>: (optional) controls the profiling of acl APIs. The value can be on (default) or off. You can profile acl API data, including the synchronous/asynchronous memory copy latencies between the host and devices and between devices.
  • --model-execution=<model-execution-value>: (optional) controls the profiling of GE model execution. The value can be on or off (default). This option will be deprecated in later versions. Use --task-time instead.
  • --runtime-api=<runtime-api-value>: (optional) controls the profiling of Runtime APIs. The value can be on or off (default). You can profile Runtime API data, including the synchronous/asynchronous memory copy latencies between the host and device and between devices.
  • --hccl=<hccl-value>: (optional) controls the communication data collection. The value can be on or off (default). The data is generated only in multi-card, multi-node, or cluster scenarios. This option will be deprecated in later versions. Use --task-time instead.
  • --task-time=<task-time-value>: (optional) controls the profiling of the operator delivery and execution durations. Related duration data must be output to files such as task_time, op_summary, and op_statistic. Possible configuration values are as follows:
    • l0: collects operator delivery and execution duration data. Compared with l1, l0 does not collect basic operator information, so the performance overhead during profiling is smaller, and this enables more accurate profiling on time consumption data.
    • l1: collects operator delivery and execution duration data, as well as basic operator information, to provide more comprehensive performance analysis data. This option supports collecting the collective communication operator data.
    • on (default): This option is enabled. This value has the same effect as l1.
    • off: This option is disabled.
  • --aicpu=<aicpu-value>: (optional) collects detailed information about AI CPU operators, such as the computation time and data copy time. The value can be on or off (default).
  • --ai-core=<aicore-value>: (optional) controls AI Core data collection. The value can be on or off. When --task-time is set to on, l1, the default value is on. When --task-time is set to off or l0, the default value is off.
  • --aic-mode=<aic-mode-value>: (optional) indicates the AI Core hardware collection type. The value can be task-based or sample-based. This option can only be used when --ai-core is set to on. In the task-based scenario, profiling is performed by task. In the sample-based scenario, profiling is performed at a fixed interval.

    You are advised to use task-based to collect AI task profile data. If this option is not set, task-based is used by default.

  • --aic-freq=<aic-freq-value>: (optional) indicates the sampling frequency in the sample-based scenario. The value ranges from 1 to 100, in Hz, and the default value is 100. This option can only be used when --ai-core is set to on.
  • --aic-metrics=<aic-metrics-value>: (optional) indicates the AI Core metrics to profile. This option can only be used when --ai-core is set to on. Possible values are:
    • ArithmeticUtilization: percentage of the time consumed by computation instructions
    • PipeUtilization: total time consumed by computation and movement instructions and the percentage of the time consumed
    • Memory: memory read/write bandwidth rate
    • MemoryL0: L0 read/write bandwidth rate
    • MemoryUB: UB read/write bandwidth rate
    • ResourceConflictRatio: resource conflict ratio
    • L2Cache: L2 cache hit ratio

      Atlas inference product: not supported

    • PipelineExecuteUtilization: total time consumed by computation and movement instructions and the percentage of the time consumed

      Atlas inference product: not supported

      Atlas training product: not supported

      Atlas A2 training product/Atlas A2 inference product: not supported

      Atlas A3 training product/Atlas A3 inference product: not supported

      Atlas 350 Accelerator Card: not supported

    • MemoryAccess:

      Atlas 200I/500 A2 inference product: not supported

      Atlas inference product: not supported

      Atlas training product: not supported

      Atlas 350 Accelerator Card: not supported

    Default value:

    Atlas 200I/500 A2 inference product: PipelineExecuteUtilization

    Atlas inference product: PipeUtilization

    Atlas training product: PipeUtilization

    Atlas A2 training product/Atlas A2 inference product: PipeUtilization

    Atlas A3 training product/Atlas A3 inference product: PipeUtilization

    Atlas 350 Accelerator Card: PipeUtilization

    You can specify the registers whose data is to be collected, for example, --aic-metrics=Custom:0x49,0x8,0x15,0x1b,0x64,0x10. The Custom field indicates the register type. It is set to specific register values in the range of [0x1, 0x7FFFFFFF]. Not all values have corresponding PMU registers. If the configured value does not have a corresponding PMU register, the profiling result may be 0. A maximum of eight registers can be configured. Separate them with commas (,). The register value can be in hexadecimal or decimal format.

  • --sys-hardware-mem=<sys-hardware-mem-value>: (optional) controls the collection of the on-chip memory read/write rate, QoS transmission bandwidth, LLC L3 cache bandwidth, accelerator bandwidth, SoC transmission bandwidth, and component memory usage. The value can be on or off (default). The collected content varies slightly depending on the model.

    Collecting memory data in the environment where glibc (earlier than 2.34) is installed may trigger a known Bug 19329. You can solve this problem by upgrading the glibc version.

  • --sys-hardware-mem-freq=<sys-hardware-mem-freq-value>: (optional) indicates the profiling frequency of --sys-hardware-mem. The value range is [1,100], in Hz, and the default value is 50.

    Atlas 350 Accelerator Card: The maximum allowed profiling frequency for QoS and SoC data is 10,000 Hz. For other items, this limit remains 100 Hz, and any configuration higher than 100 Hz will automatically fall back to 100 Hz.

    This option can only be used when --sys-hardware-mem is set to on.

    For the following products, you are advised not to increase the profiling frequency after the profiling task is complete. Otherwise, SoC transmission bandwidth data may be lost.

    Atlas 200I/500 A2 inference product

    Atlas A2 training product/Atlas A2 inference product

    Atlas A3 training product/Atlas A3 inference product

  • --l2=<l2-value>: (optional) collects the hit rates of the L2 cache and TLB page table. The value can be on or off (default).
    • Atlas A2 training product/Atlas A2 inference product: --aic-metrics=L2Cache is recommended for analyzing the number of hits on L2 from the AI Core.
    • Atlas A3 training product/Atlas A3 inference product: --aic-metrics=L2Cache is recommended for analyzing the number of hits on L2 from the AI Core.
  • --ge-api=<ge-api-value>: (optional) collects the time consumption data of dynamic shape operators in the host scheduling phase. Related data is generated in the msprof_*.json and api_statistic_*.csv files. Possible values are as follows:
    • off: (default): This option is disabled.
    • l0: collects the time consumption data of dynamic-shape operators in the main host scheduling phase to facilitate accurate statistics.
    • l1: profiles finer-grained time consumption data of dynamic-shape operators in the host scheduling phase to provide more comprehensive profile data.
  • --task-memory=<task-memory-value>: (optional) collects CANN operator memory usage for optimization. The options are as follows:
    • on: This option is enabled.
    • off (default): This option is disabled.

    In the single-operator scenario, the operator memory and lifecycle information are collected based on GE component and operator dimensions (the GE component memory is not collected in the single-operator API execution scenario). In the static graph and static subgraph scenarios, the operator memory and lifecycle information are collected based on operator dimension.

Example

For Ascend EP, log in to the operating environment and run the following command in any directory:
msprof --output=/home/projects/output --ascendcl=on --runtime-api=on --task-time=on --aicpu=on --ai-core=on /home/projects/MyApp/out/main

Find the PROF_XXX directory generated in the directory specified by --output to store the automatically parsed profile data. For details about the result files, see Profile Data File References.

For Ascend RC, log in to the operating environment, go to the /var directory where msprof is installed, and run the following command:
./msprof --output=/home/projects/output --ascendcl=on --runtime-api=on --task-time=on --aicpu=on --ai-core=on /home/projects/MyApp/out/main

The PROF_XXX directory is generated in the directory specified by --output. The files in this directory cannot be viewed without being parsed. You need to upload the PROF_XXX directory to the development environment for data parsing. For details, see Using the msprof Command to Parse, Query, and Export the Profile Data.

This site uses cookies to improve your browsing experience. By continuing to browse this site, you agree to our use of cookies.View Details