Documentation
Download

Collecting AI processor System Data

Function

msprof supports the collection of AI processor system data. After the collection, it automatically parses the profile data and flush corresponding files to disks.

Command Format

For Ascend EP, log in to the operating environment and run the following command:
msprof --output=<path> --sys-devices=<ID> --sys-period=<period> [options]
You can also collect system data when collecting the profile data during AI task running. In this case, you need to pass the user application or execution script. The --output, --sys-devices, and --sys-period options are optional.
msprof [options] <app>
For Ascend RC, log in to the operating environment, go to the /var directory where msprof is installed, and run the command.
./msprof --output=<path> --sys-devices=<ID> --sys-period=<period> [options]
You can also collect system data when collecting the profile data during AI task running. In this case, you need to pass the user application or execution script. The --output, --sys-devices, and --sys-period options are optional.
./msprof [options] <app>
  • For details about the app parameters, see app Parameters. For details about the options parameters, see Command-line Options.
  • For Ascend EP, if you use the msprof CLI to collect network-wide inference profile data and the --llc-profiling, --sys-cpu-profiling, --sys-profiling, and --sys-pid-profiling options are included, no data is profiled for any option except for --sys-cpu-profiling, which collects the TS CPU profile data. However, if no user application is passed, profiling is performed for all preceding options.
  • For the models below, --sys-profiling, --sys-pid-profiling, and --sys-cpu-profiling options cannot be used to collect data of two devices that share the same OS. For example, if there are [0, 7] devices and they share OSs in groups of 0 and 1, 2 and 3, 4 and 5, and 6 and 7, respectively, --sys-devices cannot be set to 0 and 1, 2 and 3, 4 and 5, or 6 and 7 at the same time. It can be set to 0, 2, 4, and 6, or 1, 3, 5, and 7.
    • Atlas A3 training product/Atlas A3 inference product
    • Atlas 350 Accelerator Card

Command-line Options

  • --sys-period=<sys-period-value>: This parameter is required when no user application is passed. It indicates the system sampling period (s). The value must be in the range (0, 30 × 24 × 3600]. This parameter will not be applied when a user program is passed.
  • --sys-devices=<sys-devices-value>: This parameter is required when no user application is passed. It indicates the device ID. The value can be all or multiple device IDs separated with commas (,). This parameter will not be applied when a user program is passed.
  • --ai-core=<aicore-value>: (optional) controls AI Core data collection.
  • --aic-mode=<aic-mode-value>: (optional) indicates the AI Core hardware collection type. The value can be task-based or sample-based. This option can only be used when --ai-core is set to on. In the task-based scenario, profile data is collected task by task. In the sample-based scenario, profile data is collected at a fixed interval.

    You are advised to use sample-based to collect AI processor system data. If this option is not set, sample-based is used by default.

  • --aic-freq=<aic-freq-value>: (optional) indicates the sampling frequency in the sample-based scenario. The value ranges from 1 to 100, in Hz, and the default value is 100. This option can only be used when --ai-core is set to on.
  • --aic-metrics=<aic-metrics-value>: (optional) indicates the AI Core metrics to profile. This option can only be used when --ai-core is set to on. Possible values are:
    • ArithmeticUtilization: percentage of the time consumed by computation instructions
    • PipeUtilization: total time consumed by computation and movement instructions and the percentage of the time consumed
    • Memory: memory read/write bandwidth rate
    • MemoryL0: L0 read/write bandwidth rate
    • MemoryUB: UB read/write bandwidth rate
    • ResourceConflictRatio: resource conflict ratio
    • L2Cache: L2 cache hit ratio

      Atlas inference product: not supported

    • PipelineExecuteUtilization: total time consumed by computation and movement instructions and the percentage of the time consumed

      Atlas inference product: not supported

      Atlas training product: not supported

      Atlas A2 training product/Atlas A2 inference product: not supported

      Atlas A3 training product/Atlas A3 inference product: not supported

      Atlas 350 Accelerator Card: not supported

    • MemoryAccess:

      Atlas 200I/500 A2 inference product: not supported

      Atlas inference product: not supported

      Atlas training product: not supported

      Atlas 350 Accelerator Card: not supported

    Default value:

    Atlas 200I/500 A2 inference product: PipelineExecuteUtilization

    Atlas inference product: PipeUtilization

    Atlas training product: PipeUtilization

    Atlas A2 training product/Atlas A2 inference product: PipeUtilization

    Atlas A3 training product/Atlas A3 inference product: PipeUtilization

    Atlas 350 Accelerator Card: PipeUtilization

    You can specify the registers whose data is to be collected, for example, --aic-metrics=Custom:0x49,0x8,0x15,0x1b,0x64,0x10. The Custom field indicates the register type. It is set to specific register values in the range of [0x1, 0x7FFFFFFF]. Not all values have corresponding PMU registers. If the configured value does not have a corresponding PMU register, the profiling result may be 0. A maximum of eight registers can be configured. Separate them with commas (,). The register value can be in hexadecimal or decimal format.

  • --sys-hardware-mem=<sys-hardware-mem-value>: (optional) controls the collection of the on-chip memory read/write rate, QoS transmission bandwidth, LLC L3 cache bandwidth, accelerator bandwidth, SoC transmission bandwidth, and component memory usage. The value can be on or off (default). The collected content varies slightly depending on the model.

    Specific component memory data can only be collected when AI task profiling is enabled (that means a user application is passed).

    Collecting memory data in the environment where glibc (earlier than 2.34) is installed may trigger a known Bug 19329. You can solve this problem by upgrading the glibc version.

  • --sys-hardware-mem-freq=<sys-hardware-mem-freq-value>: (optional) indicates the profiling frequency of --sys-hardware-mem. The value range is [1,100], in Hz, and the default value is 50.

    Atlas 350 Accelerator Card: The maximum allowed profiling frequency for QoS and SoC data is 10,000 Hz. For other items, this limit remains 100 Hz, and any configuration higher than 100 Hz will automatically fall back to 100 Hz.

    This option can only be used when --sys-hardware-mem is set to on.

    For the following products, you are advised not to increase the profiling frequency after the profiling task is complete. Otherwise, SoC transmission bandwidth data may be lost.

    Atlas 200I/500 A2 inference product

    Atlas A2 training product/Atlas A2 inference product

    Atlas A3 training product/Atlas A3 inference product

  • --llc-profiling=<llc-profiling-value>: (optional) indicates the LLC Profiling event. --sys-hardware-mem must be set to on. Possible values are:
    • read (default): read event, indicating the L3 cache read rate.
    • write: write event, indicating the L3 cache write rate.
  • --sys-cpu-profiling=<sys-cpu-profiling-value>: (optional) profiles CPU (such as AI CPU and Ctrl CPU) data. The value can be on or off (default).
  • --sys-cpu-freq=<sys-cpu-freq-value>: (optional) indicates the CPU profiling frequency. The value range is [1, 50], in Hz, and the default value is 50. This option can only be used when --sys-cpu-profiling is set to on.
  • --sys-profiling=<sys-profiling-value>: (optional) profiles system CPU usage and system memory. The value can be on or off (default).

    After this option is used, the Profiling tool calls the perf tool on the device. The perf tool only collects profile data and cannot obtain other runtime information. The risk is low.

  • --sys-sampling-freq=<sys-sampling-freq-value>: (optional) indicates the profiling frequency of system CPU usage and system memory. The value range is [1, 10], in Hz, and the default value is 10. This option can only be used when --sys-profiling is set to on.
  • --sys-pid-profiling=<sys-pid-profiling-value>: (optional) profiles the CPU usage and memory of all processes. The value can be on or off (default).
  • --sys-pid-sampling-freq=<sys-pid-sampling-freq-value>: (optional) indicates the profiling frequency of the CPU usage and memory of all processes. The value range is [1, 10], in Hz, and the default value is 10. This option can only be used when --sys-pid-profiling is set to on.
  • --sys-io-profiling=<sys-io-profiling-value>: (optional) profiles NIC, RoCE, MAC, and UB bandwidth data. The value can be on or off (default).
    • Atlas 200I/500 A2 inference product: profiles NIC data only in RC scenarios. This option does not take effect in container scenarios.
    • Atlas inference product: not supported.
    • Atlas training product: profiles NIC and RoCE data.
    • Atlas A2 training product/Atlas A2 inference product: profiles NIC, MAC, and RoCE data.
    • Atlas A3 training product/Atlas A3 inference product: profiles NIC, MAC, and RoCE data.
    • Atlas 350 Accelerator Card: profiles UB bandwidth data.
  • --sys-io-sampling-freq=<sys-io-sampling-freq-value>: (optional) indicates the profiling frequency of the NIC, RoCE, MAC, and UB bandwidth data. The value range is [1, 100], in Hz, and the default value is 100. This option can only be used when --sys-io-profiling is set to on.

    For the Atlas 200I/500 A2 inference product, this option is not supported.

  • --sys-interconnection-profiling=<sys-interconnection-profiling-value>: (optional) profiles HCCS, CCU bandwidth, PCIe data, inter-chip transmission bandwidth, SIO data, and UB bandwidth data. The collected content varies slightly depending on the model. The value can be on or off (default).
    • For the Atlas 200I/500 A2 inference product, this option is not supported.
    • Atlas inference product: profiles PCIe data.
    • Atlas training product: profiles HCCS and PCIe data.
    • Atlas A2 training product/Atlas A2 inference product: profiles HCCS, PCIe data, and inter-chip transmission bandwidth data.
    • Atlas A3 training product/Atlas A3 inference product: profiles HCCS, PCIe data, inter-chip transmission bandwidth, and SIO data.
    • Atlas 350 Accelerator Card: profiles CCU bandwidth, PCIe data, inter-chip transmission bandwidth, UB bandwidth, and SIO data.
  • --sys-interconnection-freq=<sys-interconnection-freq-value>: (optional) indicates the profiling frequency of the HCCS, CCU bandwidth, PCIe data, inter-chip transmission bandwidth, SIO data, and UB bandwidth data. The value range is [1, 50], in Hz, and the default value is 50.

    This option can only be used when --sys-interconnection-profiling is set to on.

    For the Atlas 200I/500 A2 inference product, this option is not supported.

  • --dvpp-profiling=<dvpp-profiling-value>: (optional) profiles the DVPP data. The value can be on or off (default).
  • --dvpp-freq=<dvpp-freq-value>: (optional) indicates the DVPP profiling frequency. The value range is [1, 100], in Hz, and the default value is 50. This option can only be used when --dvpp-profiling is set to on
  • --instr-profiling=<instr-profiling-value>: (optional) profiles the bandwidth and latency of AI Core (including AIC Core and AIV Core). The value can be on or off (default).

    This parameter is supported only by the following models:

    Atlas A2 training product/Atlas A2 inference product

    Atlas A3 training product/Atlas A3 inference product

    Atlas 350 Accelerator Card

    Specific profile data can only be collected when AI task profiling is enabled (that means, a user application is passed) in single-operator API execution scenarios.

    For Atlas 350 Accelerator Card, the statistics may be inaccurate due to the long statistical period of the last instruction segment. You are advised to use msprof op for profiling.

    This option is mutually exclusive with --ascendcl, --model-execution, --runtime-api, --hccl, --task-time, --aicpu, --ai-core, --aic-mode, --aic-freq, --aic-metrics, and --l2 and cannot be executed at the same time.

  • --instr-profiling-freq=<instr-profiling-freq-value>: (optional) indicates the profiling interval for the bandwidth and latency of the AI Core (including AIC Core and AIV Core). The value range is [300, 30000], in cycles, and the default value is 1000. Actual AI Core bandwidth and latency profiling frequency = Processor operating frequency/Value of this option. Suppose that the AI Core operating frequency is 5000 Hz and the value of this option is 1000. The profiling frequency is 5 Hz, which is 5 times per second.

    This option can only be used when --instr-profiling is set to on.

    This parameter is supported only by the following models:

    Atlas A2 training product/Atlas A2 inference product

    Atlas A3 training product/Atlas A3 inference product

Example

For Ascend EP, log in to the operating environment and run the command in any directory.
msprof --output=/home/projects/output --sys-devices=<ID> --sys-period=<period> --ai-core=on --sys-hardware-mem=on --sys-cpu-profiling=on --sys-profiling=on --sys-pid-profiling=on --dvpp-profiling=on

Find the PROF_XXX directory generated in the directory specified by --output to store the automatically parsed profile data. For details about the result files, see Profile Data File References.

For Ascend RC, log in to the operating environment, go to the /var directory where msprof is installed, and run the command.
./msprof --output=/home/projects/output --sys-devices=<ID> --sys-period=<period> --ai-core=on --sys-hardware-mem=on --sys-cpu-profiling=on --sys-profiling=on --sys-pid-profiling=on --dvpp-profiling=on

The PROF_XXX directory is generated in the directory specified by --output. The files in this directory cannot be viewed without being parsed. You need to upload the PROF_XXX directory to the development environment for data parsing. For details, see Using the msprof Command to Parse, Query, and Export the Profile Data.

This site uses cookies to improve your browsing experience. By continuing to browse this site, you agree to our use of cookies.View Details