General Description

After raw profile data is collected, parsed, and exported into visualized profile data files, the file directory structure and main files are as follows:

Directory Structure and File Description

The following is an example of the structure of a profile data directory (only profile data is displayed):

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
├── msprof_*.db
├── mindstudio_profiler_output
│    ├── msprof_*.json
│    ├── step_trace_*.json
│    ├── xx_*.csv
...
│    └── README.txt 
├── device_{id}
...
│    └── data
├── host
...
      └── data
The asterisk (*) indicates the timestamp.
  • The mindstudio_profiler_output directory stores the profile data summary of the host and each device. (You are advised to view the files in this directory for profile data analysis.) The result files in the directory are named in the format of module-name_{timestamp}.{json/csv}.
  • The device_{id} directory stores the raw profile data of Ascend AI applications running on each device and the raw system data of AI processor.
  • The host directory stores the raw profile data of Ascend AI application running of upper-layer application APIs (msproftx) and raw system data on the host.

The data volume of each file in this directory is relatively large, and data file slicing may occur. If file slicing is not required, refer to Profile Data File Slicing to modify the upper limit of the slicing capacity.

Table 1 shows the profile data files collected by default.

Table 1 Profile data files collected by msprof by default

File Name

Description

msprof_*.db

.db file that aggregates all profile data.

msprof_*.json

Timeline report.

step_trace_*.json

Iteration trace data, which records the time required for each iteration. This profile data file does not exist in single-operator scenarios (such as the PyTorch scenario).

op_summary_*.csv

AI Core and AI CPU operator data.

op_statistic_*.csv

Number of times that the AI Core and AI CPU operators are called and the durations.

step_trace_*.csv

Iteration trace data. This profile data file does not exist in single-operator scenarios (such as the PyTorch scenario).

task_time_*.csv

Task Scheduler data.

fusion_op_*.csv

Operator fusion summary in a model. This profile data file does not exist in single-operator scenarios (such as the PyTorch scenario).

api_statistic_*.csv

Time spent by API execution at the CANN layer.

The .json files in the table are timeline information files, which collect the running durations of operators and tasks and display the data in color blocks. The .csv files are summary information files, which summarize the running durations in tables.

Viewing Overall Profile Data

The msprof_*.db file summarizes the overall profile data. The timeline and summary files in the mindstudio_profiler_output directory split the profile data into multiple files, the preceding two types of files can be imported using MindStudio Insight to display the overall data in a visualized manner. For details, see MindStudio Insight User Guide.

Viewing the Timeline File

Use the Perfetto UI to open a file: Enter https://ui.perfetto.dev/ in the address box of Google Chrome, drag the .json file to the blank space, and press the shortcut keys (w: zoom in; s: zoom out; a: move left; d: move right) on the keyboard to view the file.

Use tracing to open a file: Enter chrome://tracing in the address box of Google Chrome, drag the .json file to the blank space, and press the shortcut keys (w: zoom in; s: zoom out; a: move left; d: move right) on the keyboard to view the file.

You are advised to use the Perfetto UI to open oversized files.

Summary File Description

Parent topic: Profile Data File References