Output Description

After the msLeaks tool analyzes the memory, it outputs files as listed in Table 1.

Table 1 Output file description

Output File Name

Description

leaks_dump_{timestamp}.csv

When the memory analysis function in a step is used, the memory information result file is generated and saved in the leaksDumpResults/dump directory by default. For details, see leaks_dump_{timestamp}.csv File Description.

{device}_trace_{timestamp}.json

When the memory analysis function in a step is used, the visualization memory information result file is generated and saved in the leaksDumpResults/trace directory by default. For details, see {device}_trace_{timestamp}.json File Description.

NOTE:

In the next version of MindStudio, the .json file will not be saved to the path, and the results will be printed to the screen.

memory_compare_{timestamp}.csv

When the memory comparison function is used, the memory comparison result file is generated, recording the baseline, comparison, and differences after comparison. The output file is saved in the leaksDumpResults/compare directory by default. For details, see memory_compare_{timestamp}.csv File Description.

leaks_dump_{timestamp}.db

The memory information result file in .db format can be displayed using MindStudio Insight. For details about the display result and operations, see "Memory Tuning" in MindStudio Insight User Guide.

python_trace_{TID}_{timestamp}.csv

The collection result file using Python Trace is saved in leaksDumpResults/dump by default. For details, see python_trace_{TID}_{timestamp}.csv File Description.

leaks_dump_{timestamp}.csv File Description

Table 2 describes the result file fields of the memory leak detection.
Table 2 Fields and meanings of leaks_dump_{timestamp}.csv

Field

Description

ID

Event ID.

Event

Event types recorded by msLeaks, including:

  • SYSTEM: system-level event
  • MALLOC: memory allocation
  • FREE: memory deallocation
  • ACCESS: memory access
  • OP_LAUNCH: operator execution
  • KERNEL_LAUNCH: kernel execution
  • MSTX: instrumentation

Event Type

Event subtypes.

  • When Event is SYSTEM, Event Type includes ACL_INIT and ACL_FINI.
  • When Event is MALLOC or FREE, Event Type includes HAL, PTA, MindSpore, ATB, HOST, and PTA_WORKSPACE.
  • When Event is ACCESS, Event Type includes READ, WRITE, and UNKNOWN.
  • When Event is OP_LAUNCH, Event Type includes ATEN_START, ATEN_END, ATB_START, and ATB_END.
  • When Event is KERNEL_LAUNCH, Event Type includes KERNEL_LAUNCH, KERNEL_START, and KERNEL_END.
  • When Event is MSTX, Event Type includes Mark, Range_start, and Range_end.

Name

The value of Name depends on the value of Event. When the value of Event is any of the following, the value of Name has different meanings. If the value of Event is not one of the preceding values, the value of Name is N/A.

  • ACCESS: The value of Name is the name or ID of the operator that triggers the access.
  • OP_LAUNCH: The value of Name is the operator name.
  • KERNEL_LAUNCH: The value of Name is the kernel name.
  • MSTX: The value of Name is a user-defined instrumentation name.

Timestamp(ns)

Time when an event occurs.

Process ID

Process information.

Thread ID

Thread information.

Device ID

Device information.

Ptr

Memory address, which can be used as the ID of a memory block. The lifecycle of a memory block is from malloc of the same ptr to the next free.

Attr

Event-specific attributes. Each event type has its own attribute. The specific display information is as follows:

  • When Event is MALLOC or FREE, the following parameters are displayed:
    • allocation_id: Same allocation_id indicates that the operations are performed on the same memory block.
    • addr: address
    • size: size of the memory allocated or freed this time
    • owner: memory block owner. The format is {A}@{B}@{C} when multi-level classification is used. This parameter is available only when Event is MALLOC.
    • total: total size of the memory pool. This parameter is available only when Event Type is PTA, MindSpore, or ATB.
    • used: total size of secondary allocation of the memory pool. This parameter is available only when Event Type is PTA, MindSpore, or ATB.
    • inefficient: indicates whether the memory is inefficient. The value indicates the inefficiency category: early_allocation indicates that the allocation is too early; late_deallocation indicates that the deallocation is too late; and temporary_idleness indicates temporary idleness. This parameter is available only when Event is MALLOC and Event Type is PTA or ATB.
  • When Event is ACCESS, the following parameters are displayed:
    • dtype: tensor dtype
    • shape: tensor shape
    • size: tensor size
    • format: tensor format
    • type: memory pool type like ATB
    • allocation_id: The same allocation_id indicates that operations are performed in the same memory. This parameter is available only when Event Type is PTA.
  • When Event is OP_LAUNCH and Event Type is ATB_START or ATB_END, the following parameters are displayed:
    • path: location of the operator in the model, for example, 0_1967120/0/0_GraphOperation/0_ElewiseOperation. The value contains pid, corresponding module name, and operator name.
    • workspace ptr: start address of the workspace memory.
    • workspace size: workspace memory size.
  • When Event is KERNEL_LAUNCH, the following parameters are displayed:
    • path: location of the kernel in the model, for example, 0_1967120/1/0_GraphOperation/1_ElewiseOperation/0_AddF16Kernel/before. The path contains pid, corresponding operator name, and kernel name. This parameter is available only when Event Type is KERNEL_START or KERNEL_END.
    • streamId: stream ID
    • taskId: task ID

Call Stack(Python)

(Optional) Python call stack information.

Call Stack(C)

(Optional) C call stack information.

{device}_trace_{timestamp}.json File Description

The memory visualization output consists of two types of .json files, including cpu_trace_{timestamp}.json (visualization file on the host) and npu{id}_trace_{timestamp}.json (visualization file on the device), where id indicates the device ID, for example, npu0_trace_{timestamp}.json.

Files can be displayed by using visualization tools, such as Chrome trace (chrome://tracing/) or Perfetto (Perfetto UI), as shown in Figure 1.

Figure 1 Visualization information on the host
Figure 2 Visualization information on the device
Table 3 provides details about the visualization information on the host and device. The visualization files and mstx instrumentation information can be used to help locate memory problems and determine the code lines where the problems occur.
Table 3 Visualization information details

File

Name

Description

cpu_trace_{timestamp}.json (visualization file on the host)

Process PID

Kernel launch instrumentation information. PID in the name indicates the process ID.

memory size

CPU memory information. This parameter is collected only when the memory collection function on the host is enabled.

pin memory size

Host pinned memory allocated on the hal layer.

npu{id}_trace_{timestamp}.json (visualization file on the device)

Process PID

Kernel launch instrumentation information. PID in the name indicates the process ID.

{Framework Name} allocated memory size

Allocated information of the memory pool of the corresponding framework.

{Framework Name} reserved memory size

Reserved information of the memory pool of the corresponding framework.

Thread TID

Duration of each step. TID in the name indicates the thread ID.

mstx 0

mstx instrumentation information.

Thread 0

Information of memory between steps that is not deallocated for a long time.

memory_compare_{timestamp}.csv File Description

Table 4 describes the result file fields of the memory comparison.

Table 4 memory_compare_{timestamp}.csv field description

Field

Description

Event

The comparison event type recorded by msLeaks. The value can be OP_LAUNCH or KERNEL_LAUNCH.

Name

Kernel name.

Device ID

Device type and card number.

Base

Data in the first input file path.

Compare

Data in the second input file path.

Allocated Memory(byte)

Memory changes before and after the kernel call.

If the value is N/A, the kernel is not called.

Diff Memory(byte)

Relative memory changes of Base and Compare.

  • If the value is 0, the memory changes caused by the kernel call are the same.
  • If the value is not 0, the memory changes caused by the kernel call are different.

python_trace_{TID}_{timestamp}.csv File Description

Table 5 describes the fields in the Python Trace result file.
Table 5 python_trace_{TID}_{timestamp}.csv field description

Field

Description

FuncInfo

Function name.

StartTime(ns)

Start timestamp, which is the same as the event timestamp in the leaks_dump_{timestamp}.csv file.

EndTime(ns)

End timestamp.

Thread ID

Thread information.

Process ID

Process information.