operator_memory (Details About Memory Usage of CANN Operators)
The memory usage details of CANN operators do not contain the timeline information. The summary information is summarized in the operator_memory_*.csv file, which records the memory required for executing a specific CANN-level operator on the NPU and the occupation time.
Applicability
Product |
Supported |
|---|---|
Atlas 350 Accelerator Card |
√ |
√ |
|
√ |
|
√ |
|
√ |
|
√ |
|
Atlas 350 Accelerator Card |
√ |
Data Description of the operator_memory_*.csv File
The following shows the file content format.
See the following table for more details.
Field |
Description |
|---|---|
Device_id |
Device ID. |
Name |
Operator name. |
Size(KB) |
Size of the memory occupied by the operator, in KB. |
Allocation Time(us) |
Memory allocation time, in μs. |
Duration(us) |
Memory occupation time, in μs. |
Allocation Total Allocated(KB) |
Total allocation of the GE memory pool during operator memory allocation, in KB. |
Allocation Total Reserved(KB) |
Total amount of the GE memory pool during operator memory allocation, in KB. |
Release Total Allocated(KB) |
Total allocation of the GE memory pool during operator memory deallocation, in KB. |
Release Total Reserved(KB) |
Total amount of the GE memory pool during operator memory deallocation, in KB. |
Device |
Device type and device ID. Only NPUs are involved. |
Negative and Empty Value Description
The reason for the presence of empty or negative values in some information in the operator_memory_*.csv file is that some operators' memory allocation or deallocation processes may not have been captured by the profile data collection process, as they occurred outside the scope. For details, see the following example.
Negative value description: In the preceding figure, the Size column in row 4873 shows a negative value. (The memory allocation size is positive, while the memory deallocation size is negative. If memory is allocated and deallocated within the scope of profile data collection, then Size takes the allocation value.) However, the Name column cannot identify the operator name, the other Allocation columns are all empty, and the Release columns have normal memory deallocation values. This indicates that memory allocation of the operator occurred before the profile data collection process, but memory deallocation occurred within the scope of profile data collection, resulting in only a negative value being collected for memory deallocation. Furthermore, the operator name can be identified only during memory allocation, so it is not possible to identify it during memory deallocation. Additionally, since memory allocation is not within the scope of profile data collection, the Allocation column shows empty memory allocation.
Empty value description: In the preceding figure, for operators starting from row 4874, the Release columns are empty while other values are normal. This indicates that while the memory allocation for these operators occurred during the profiling process, the memory deallocation occurred outside this period, resulting in empty entries in the Release columns.