Overview

During performance tuning, the profiling tools can be used to collect and analyze key performance metrics of AI jobs at different stages during their execution on Ascend AI Processors. You can efficiently locate software and hardware performance bottlenecks based on the output profile data, thereby enhancing the overall efficiency of AI job performance analysis.

Quick Start

For offline inference, use the msprof CLI for profiling. For details, see Getting Started with Performance Analysis in Offline Inference Scenarios. If the CANN Toolkit package and ops operator package have not been installed in the current environment, the msprof CLI is unavailable.
For training, modify the API parameters in the AI framework for profiling. For details, see Getting Started with Performance Analysis in PyTorch Training Scenarios and Getting Started with Performance Analysis in TensorFlow Training Scenarios.

Tool Navigation

After using the msprof CLI or the PyTorch framework APIs to collect profile data, you do not need to use the msprof CLI to parse the data because the data can be automatically parsed. For other collection methods, you need to use the msprof CLI to parse the profile data.

**Table 1** Tool navigation
Scenario	Tools	Description
General-purpose scenarios	msprof	The msprof CLI can profile more types of data and analyze the profile data. The Python call stack, PyTorch or MindSpore framework layer data cannot be profiled. You can use the framework APIs to profile such data.
General-purpose scenarios	MSPTI	MSPTI is designed for general-purpose scenarios. Tools built with MSPTI APIs can profile inferenceand training in various frameworks.
MindIE Service inference serving	msServiceProfiler	The msServiceProfiler APIs collect start and end times for key processes, identify main functions or iterations, and record important events during the MindIE Service inference serving. They profile various data and help pinpoint performance issues quickly.
MindSpore framework	MindSpore Profiler	This tool is used for programming in the MindSpore framework.
PyTorch framework	Ascend PyTorch Profiler	This tool is used for programming in the PyTorch framework. Restriction: This tool works only for training and online inference when you need to call the profiling code during programming in the AI framework.
TensorFlow framework	Using TensorFlow APIs for Data Profiling	This tool is used for programming in the TensorFlow framework. Restriction: This tool works only for training and online inference when you need to call the profiling code during programming in the AI framework.
TensorFlow framework	Profiling with Environment Variables	Set specific environment variables to control profiling. Migrate these configurations to different scripts for training or online inference. Restriction: This tool works only for training and online inference.
Graph development	Using Ascend Graph APIs for Data Profiling	Use this tool during Ascend Graph development. Restriction: This tool works only in training and online inference when you need to call the profiling APIs during Ascend Graph compilation.
Offline inference	Using acl (Adapted for C & C++) APIs for Data Profiling	This is the most flexible tool for custom data profiling. Restriction: This tool works only in offline inference when you need to call the profiling APIs in your application.
Offline inference	Using acl (Adapted for Python) APIs for Data Profiling	This is the Python wrapper for acl APIs. Restriction: This tool works only in offline inference when you need to call the profiling APIs in your application.
Offline inference	Using the acl.json Configuration File for Data Profiling	This is a configuration file where you can set profiling along with other components. Restriction: This works only for offline inference when you need to modify the configuration file.