Using the acl.json Configuration File for Profiling

In the offline inference scenario, you can call the acl.json file in the inference application to read the profiling parameters to automatically collect raw profile data.

After that, you can analyze the collected profile data in the development environment where the tool is installed and view the analysis results. For details about parsing operations, see Using the msprof Command to Parse, Query, and Export the Profile Data. For details about parsing result files, see Profile Data File References.

This section describes only how to enable data profiling in the inference application. For details about the complete development process of the inference application, see Application Development (C&C++).

Prerequisites

The inference application can run properly before you enable data profiling.
You have called aclInit() and aclFinalize() for initialization and deinitialization.

Procedure

Configure the acl.json file, and build and run the application project by taking the following steps:

Open the code file of the inference application project where the aclInit() function is located and obtain the path of the acl.json file.

// ACL init
const char *aclConfigPath = "../src/acl.json";
aclError ret = aclInit(aclConfigPath);
if (ret != ACL_ERROR_NONE) {
    ERROR_LOG("acl init failed");
    return FAILED;
}
INFO_LOG("acl init success");

If the acl.json file path is not passed to the aclInit() call, modify the function and pass the path created in Step 2.

Modify the acl.json file in the directory (if the file does not exist, create it in the src directory after project build) and add the related Profiling configuration in the following format.
1 2 3 4 5 6
{ "profiler": { "switch": "on", "output": "output" } }
- switch: profiling switch, either on or off.
  If this parameter is not included or is not set to on, profiling is disabled.
  
  After profiling is enabled, the Runtime APIs and Task Scheduler data are automatically collected.
- output (optional): path for flushing profile data to the disk. If this parameter is not set, the profile data is flushed to the directory where the executable file of the application project is located by default.
  The path cannot contain the following characters: "\n", "\\n", "\f", "\\f", "\r", "\\r", "\b", "\\b", "\t", "\\t", "\v", "\\v", "\u007F", "\\u007F", "\"", "\\\"", "'", "\'", "\\", "\\\\", "%", "\\%", ">", "\\>", "<", "\\<", "|", "\\|", "&", "\\&", "$", "\\$", ";", "\\;", "`", "\\`".
  After profiling is complete, directories starting with PROF are generated in this specified directory and will store the raw profile data. The path can be an absolute path or a relative path (relative to the path where commands are executed).
  - An absolute path starts with a slash (/).
  - A relative path starts with a directory name, for example, output.
  - Ensure that the running user configured during installation has the read and write permissions on the directory specified by this option. If the user does not have the read and write permissions on this directory, the profile data will be stored in the path of the executable file by default (ensure that the running user has the read and write permissions on this default path).
  - This option has a higher priority than ASCEND_WORK_PATH. For details, see the Environment Variables.
- storage_limit: maximum size of files that can be stored in a specified disk directory. If the size of profile data files in the disk is about to use up the maximum storage space specified by this option or the total remaining disk space is about to be used up (remaining space ≤ 20 MB), the earliest files in the disk are aged and deleted.
  The value range is [200, 4294967295], in MB, for example, storage_limit=200MB. By default, this option is not set.
  
  If this parameter is not set and the available disk space is less than 20 MB before profiling, data is not flushed to the disk.
- aicpu: details about the AI CPU operator, such as the operator execution time and data copy time. The value can be on or off (default).
- aic_metrics: AI Core performance metrics. This parameter takes effect only when task_time or task_trace is set to on or l1. If task_time or task_trace is set to l0 or off, profiling specified by this parameter is not executed. Possible values are:
  - ArithmeticUtilization: percentage of time consumed by computation instructions
  - PipeUtilization: total time and percentage of time used by computation and movement instructions
  - Memory: memory read/write bandwidth rate
  - MemoryL0: L0 read/write bandwidth rate
  - MemoryUB: UB read/write bandwidth rate
  - ResourceConflictRatio: resource conflict ratio
  - L2Cache: L2 cache hit ratio
    Atlas inference product: not supported
  - PipelineExecuteUtilization: total time consumed by computation and movement instructions and the percentage of the time consumed
    Atlas inference product: not supported
    
    Atlas training product: not supported
    
    Atlas A2 training product/Atlas A2 inference product: not supported
    
    Atlas A3 training product/Atlas A3 inference product: not supported
    
    Atlas 350 Accelerator Card: not supported
  - MemoryAccess:
    Atlas 200I/500 A2 inference product: not supported
    
    Atlas inference product: not supported
    
    Atlas training product: not supported
    
    Atlas 350 Accelerator Card: not supported
  Default value:
  
  Atlas 200I/500 A2 inference product: PipelineExecuteUtilization
  
  Atlas inference product: PipeUtilization
  
  Atlas training product: PipeUtilization
  
  Atlas A2 training product/Atlas A2 inference product: PipeUtilization
  
  Atlas A3 training product/Atlas A3 inference product: PipeUtilization
  
  Atlas 350 Accelerator Card: PipeUtilization
  You can specify the registers whose data is to be collected, for example, "aic_metrics":"Custom:0x49,0x8,0x15,0x1b,0x64,0x10".
  - The Custom field indicates the register type. It is set to specific register values in the range of [0x1, 0x7FFFFFFF]. Not all values have corresponding PMU registers. If the configured value does not have a corresponding PMU register, the profiling result may be 0.
  - A maximum of eight registers can be configured. Separate them with commas (,).
  - The register value can be in hexadecimal or decimal format.
- l2: switch that controls L2 cache hit ratio and TLB page table cache hit ratio, either on or off (default).
  - Atlas 200I/500 A2 inference product: profiles the L2 cache hit ratio. aic-metrics=L2Cache is recommended for analyzing the number of hits on L2 cache from the AI Core.
  - Atlas inference product: profiles the L2 cache hit ratio.
  - Atlas training product: profiles the L2 cache hit ratio.
  - Atlas A2 training product/Atlas A2 inference product: profiles the L2 cache hit ratio and TLB page table cache hit ratio. aic-metrics=L2Cache is recommended for analyzing the number of hits on L2 cache from the AI Core.
  - Atlas A3 training product/Atlas A3 inference product: profiles the L2 cache hit ratio and TLB page table cache hit ratio. aic_metrics=L2Cache is recommended for analyzing the number of hits on L2 cache from the AI Core.
  - Atlas 350 Accelerator Card: profiles the L2 cache hit ratio and TLB page table cache hit ratio. aic_metrics=L2Cache is recommended for analyzing the number of hits on L2 cache from the AI Core.
- hccl: communication data profiling switch. The data is generated only in multi-card, multi-node, or cluster scenarios.
  - This parameter can be set to on or off in the .json file.
  - If this parameter is not set in the .json file, the data is not collected by default. When task_time is set to on, this parameter is automatically set to on.
  This switch will be deprecated in later versions. Use the task_time switch to control related profiling.
- task_time: switch that controls profiling of the operator delivery and execution durations. Related duration data must be output to files such as task_time, op_summary, and op_statistic. Possible configuration values are as follows:
  - on (default): enabled
  - off: This option is disabled.
  - l0: collects operator delivery and execution duration data. Compared with l1, l0 does not collect basic operator information, so the performance overhead during profiling is smaller, and this enables more accurate profiling of statistics on time duration data.
  - l1: collects operator delivery and execution duration data, as well as basic operator information, to provide more comprehensive performance analysis data. The effect is the same as that when this parameter is set to on.
- ge_api: switch that controls profiling of the time consumption data of dynamic-shape operators in the host scheduling phase. Possible values are as follows:
  - off (default): disabled
  - l0: collects the time consumption data of dynamic-shape operators in the main host scheduling phase to facilitate accurate statistics.
  - l1: profiles finer-grained time consumption data of dynamic-shape operators in the host scheduling phase to provide more comprehensive profile data.
- ascendcl: acl API profiling switch, either on (default) or off. You can profile acl API data, including the synchronous/asynchronous memory copy latencies between the host and devices and between devices.
- runtime_api: runtime API profiling switch, either on (default) or off. You can profile Runtime API data, including the synchronous/asynchronous memory copy latencies between the host and device and between devices.
- sys_hardware_mem_freq: switch that controls the profiling frequency of the on-chip memory read/write speed, QoS transmission bandwidth, LLC L3 cache bandwidth, accelerator bandwidth, SoC transmission bandwidth, and component memory usage. The collected content varies slightly depending on the product. The value range is [1, 100], in Hz. By default, data is not collected.
  Atlas 350 Accelerator Card: The maximum allowed profiling frequency for QoS and SoC data is 10,000 Hz. For other items, this limit remains 100 Hz, and any configuration higher than 100 Hz will automatically fall back to 100 Hz.
  
  Collecting memory data in the environment where glibc (earlier than 2.34) is installed may trigger a known Bug 19329. You can solve this problem by upgrading the glibc version.
  
  For the following products, you are advised not to increase the profiling frequency after the profiling task is complete. Otherwise, SoC transmission bandwidth data may be lost.
  
  Atlas 200I/500 A2 inference product
  
  Atlas A2 training product/Atlas A2 inference product
  
  Atlas A3 training product/Atlas A3 inference product
- llc_profiling: LLC profiling event. To collect the data, you need to set sys_hardware_mem_freq. Possible values are:
  - read (default): read event, indicating the L3 cache read rate
  - write: write event, indicating the L3 cache write rate
- sys_io_sampling_freq: NIC, UB bandwidth, and ROCE profiling frequency. The collected content varies slightly depending on the product. The value range is [1, 100], in Hz.
  - Atlas 200I/500 A2 inference product: profiles NIC data only in RC scenarios. This option does not take effect in container scenarios.
  - Atlas inference product: not supported.
  - Atlas training product: profiles NIC and RoCE data.
  - Atlas A2 training product/Atlas A2 inference product: profiles NIC and RoCE data.
  - Atlas A3 training product/Atlas A3 inference product: profiles NIC and RoCE data.
  - Atlas 350 Accelerator Card: profiles UB bandwidth data.
- sys_interconnection_freq: profiling frequency for HCCS, CCU bandwidth, SIO data, PCIe data, inter-chip transmission bandwidth, and UB bandwidth. The collected content varies slightly depending on the product.
  The value range is [1, 50], in Hz. By default, data is not collected.
  - For the Atlas 200I/500 A2 inference product, this option is not supported.
  - Atlas inference product: profiles PCIe data.
  - Atlas training product: profiles HCCS and PCIe data.
  - Atlas A2 training product/Atlas A2 inference product: profiles HCCS, PCIe data, and inter-chip transmission bandwidth data.
  - Atlas A3 training product/Atlas A3 inference product: profiles HCCS, PCIe data, inter-chip transmission bandwidth, and SIO data.
  - Atlas 350 Accelerator Card: profiles PCIe data, inter-chip transmission bandwidth, SIO, UB bandwidth, and CCU bandwidth.
- dvpp_freq: DVPP collection frequency. The value range is [1, 100], in Hz.
- instr_profiling: profiling frequency of the bandwidth and latency for AI Core and AI Vector.
  It is supported only in single-operator API execution scenarios. It is supported only by the Atlas 350 Accelerator Card.
  
  The statistics may be inaccurate due to the long statistical period of the last instruction segment. You are advised to use msprof op for profiling.
- instr_profiling_freq: profiling frequency of the bandwidth and latency for AI Core and AI Vector. The value range is [300, 30000], in Hz.
  It is supported only in single-operator API execution scenarios.
  
  This switch is mutually exclusive with aicpu, aic_metrics, l2, hccl, task_time, ascendcl, and runtime_api, and cannot be executed at the same time.
  
  This parameter is supported only by the following models:
  
  Atlas A2 training product/Atlas A2 inference product
  
  Atlas A3 training product/Atlas A3 inference product
- host_sys: host-side profiling. Possible values include:
  - cpu: process CPU usage
  - mem: process memory usage
  - disk: process disk I/O usage
  - network: network I/O usage
  - osrt: process syscall and pthreadcall
  You can select one or more options and separate them with commas (,), for example, "host_sys": "cpu,mem".
- host_sys_usage: CPU and memory data of the system and all process on the host. The values are cpu and mem. You can select one or more options and separate them with commas (,), for example, "host_sys_usage": "cpu,mem".
- host_sys_usage_freq: profiling frequency of the CPU and memory data of the system and all processes on the host. The value range is [1, 50], in Hz, and the default value is 50.
- msproftx: controls the msproftx user and upper-layer framework program to output profile data, either on or off (default).
  Before enabling msproftx for profiling, you need to call msproftx APIs in the program to enable the output of profile data streams and record the time span of specific events during application execution.
After the acl.json file is configured, rebuild and run the application project. For details, see the Application Development (C&C++).
output specifies the path for storing collected profile data, as shown in Figure 1.

Figure 1 Profile data of the application project

If the acl.json file already exists, modify the file content and add profiling configurations. You do not need to rebuild the application project.

Parent topic: Other Profiling Methods