Quick Analysis for Model Tuning (msprof-analyze CLI)
msprof-analyze provides a command line tool for quick analysis of performance bottlenecks in AI jobs. Table 1 lists the three core capabilities.
|
Tool |
Function Description |
|---|---|
|
cluster_analyze (cluster analysis) |
Locates slow nodes, cards, and links. It can be used together with MindStudio Insight. |
|
compare (performance breakdown and comparison) |
Compares operators between NPUs and GPUs and between two NPUs in terms of time and memory, helping users quickly locate operators. |
|
Advisor (expert suggestions) |
Based on the experience of performance optimization experts and the affinity adaptation of Ascend software and hardware to operators, the automatic tuning capability is provided to help users identify performance bottlenecks and provide tuning suggestions. |
- cluster_analyze (cluster analysis)
The cluster_analyze result is displayed using MindStudio Insight to help analyze the communication matrix and communication duration.
Figure 1 Visual cluster analysis result on MindStudio Insight
- compare tool for performance comparison
The compare tool divides the duration into four core dimensions: operator execution, communication (not overlapped with computing), scheduling overhead, and memory usage, enabling precise identification of performance bottlenecks.
Figure 2 Analysis result report of the compare tool
- Advisor expert suggestion
Advisor automatically identifies performance bottlenecks and provides tuning suggestions. It covers the delivery, computing, and communication dimensions in cluster and single-card scenarios, analyzing profile data from end to end.
Figure 3 Main functions of the Advisor tool
The Advisor tool classifies tuning suggestions by urgency. The suggestions marked in red have the highest priority and need to be handled first.
Figure 4 Analysis result report of the Advisor tool