msPTI

Overview

MindStudio Profiling Tool Interface (msPTI) is a set of profiling APIs provided by MindStudio for Ascend devices. You can use msPTI to build profilers for NPU applications.

msPTI is for common scenarios. Profiling tools developed with msPTI APIs can be used in inferencetraining scenarios in different frameworks.

msPTI provides the following functions:

Tracing: In msPTI, tracing refers to the collection of timestamps and additional information when CANN applications start activities, including information about CANN APIs, kernels, and memory copies. Identify performance issues of CANN by understanding the program running duration. You can use the Activity APIs and Callback APIs to collect tracing information.
Profiling: In msPTI, profiling refers to the collection of NPU performance metrics of a kernel or a group of kernels.

msPTI provides a set of C-based APIs and a set of Python APIs with the C-based API functions as the underlying logic.

Applicability

For details about the mapping between AI processors and Ascend products, see Ascend Product Form Description.

Product	Supported
Atlas 350 Accelerator Card	√
Atlas A3 training product/Atlas A3 inference product	√
Atlas A2 training product/Atlas A2 inference product	√
Atlas 200I/500 A2 inference product	√
Atlas inference product	x
Atlas training product	x

Preparations

The samples of msPTI Python APIs depend on the PyTorch framework and torch_npu plugin. Ensure that they have been installed. For details, see "Installing PyTorch" in the Ascend Extension for PyTorch.

Restrictions

msPTI cannot be used with any other profilers. Otherwise, the profile data will be lost.

msPTI Sample Set

This section provides examples of using msPTI APIs.

Sample Building and Execution

After installing the CANN software, log in to the environment as the CANN running user and run the following command to configure environment variables: ${install_path} indicates the path for storing files after the CANN software is installed, for example, /usr/local/Ascend/cann. The following is an example:
```
source /usr/local/Ascend/cann/set_env.sh
```
Go to the sample directory.
The msPTI sample code is integrated in the CANN Toolkit development kit and stored in the ${install_path}/tools/mspti/samples directory.

Replace ${install_path} with the path for storing files after the CANN software is installed. For example, if the installation is performed by the root user, the path is /usr/local/Ascend/cann.
Example:
```
cd ${install_path}/tools/mspti/samples/callback_domain
```
Execute sample_run.sh in the sample directory.
```
bash sample_run.sh
```

The following table describes the samples provided currently.

Callback APIs

Sample	Description
callback_domain	Demonstrates the Callback API function. You can call msptiEnableDomain to perform callback operations before and after the runtime API.
callback_mstx	Demonstrates the function of combining the callback and mstx APIs. You can use the callback and mstx APIs to profile operator data before and after the runtime launch kernel. Demonstrates the usage of userdata in callback. You can use userdata to transparently transmit configurations or specific running parameters.

Sample

Description

callback_domain

Demonstrates the Callback API function. You can call msptiEnableDomain to perform callback operations before and after the runtime API.

callback_mstx

Demonstrates the function of combining the callback and mstx APIs. You can use the callback and mstx APIs to profile operator data before and after the runtime launch kernel.

Demonstrates the usage of userdata in callback. You can use userdata to transparently transmit configurations or specific running parameters.

Activity APIs

Sample	Description
mspti_activity	Demonstrates the basic functions of the Activity API. It shows how to profile kernels and memory. Demonstrates the basic running of the Activity API and describes the basic usage of the Activity API, including the memory allocation of the activity buffer and the logic of buffer consumption.
mspti_correlation	Demonstrates the basic functions of the Activity API. It shows how to use the correlationId field to correlate the API with the kernel data. Demonstrates the correlation between the runtime API delivery and the actual kernel execution data. After the association, the delivery and execution of operators can be one-to-one mapped, which facilitates performance bottleneck analysis.
mspti_external_correlation	Demonstrates the msPTI external correlation function. Demonstrates the usage of msptiActivityPopExternalCorrelationId and msptiActivityPushExternalCorrelationId. You can use it to correlate various APIs to trace function call stacks.
mspti_hccl_activity	Demonstrates the basic functions of the Activity APIs and shows how to use the HCCL switch to collect communication data.
mspti_mstx_activity_domain	Demonstrates how msPTI controls the mstxDomain function. You can enable or disable the function to control domain profiling. You can use the msPTI switch to enable or disable profiling in real time, reducing performance loss.

Python APIs

Sample	Description
python_monitor	Demonstrates the basic usage of Monitor. You can use KernelMonitor and HcclMonitor to obtain the time consumed by compute operators and communication operators.
python_mstx_monitor	Demonstrates the basic usage of MstxMonitor. You can use mstx to collect the time consumed by a specific operator (for example, matmul).

Parent topic: Other Profiling Methods