operator_memory (Details About Memory Usage of CANN Operators)
The memory usage details of CANN operators do not contain the timeline information. The summary information is summarized in the operator_memory_*.csv file, which records the memory required for executing a specific CANN-layer operator on the NPU and the occupation time.
Availability
Atlas 200/500 A2 Inference Product
Atlas Inference Series Product
Atlas Training Series Product
Atlas A2 Training Series Product/Atlas 800I A2 Inference Product
Atlas A3 Training Series Product
Data Description of the operator_memory_*.csv File
The file content is formatted as follows.
See the following table for more details.
Field |
Description |
|---|---|
Device_id |
Device ID. |
Name |
Operator name. |
Size(KB) |
Size of the memory occupied by the operator, in KB. |
Allocation Time(us) |
Memory allocation time, in μs. |
Duration(us) |
Memory occupation time, in μs. |
Allocation Total Allocated(KB) |
Total allocation of the GE memory pool during operator memory allocation, in KB. |
Allocation Total Reserved(KB) |
Total amount of the GE memory pool during operator memory allocation, in KB. |
Release Total Allocated(KB) |
Total allocation of the GE memory pool during operator memory deallocation, in KB. |
Release Total Reserved(KB) |
Total amount of the GE memory pool during operator memory deallocation, in KB. |
Device |
Device type and device ID. Only NPUs are involved. |
Negative and Empty Value Description
Some fields in the operator_memory_*.csv file are empty or negative because the corresponding memory allocation or deallocation events for certain operators fall outside the scope of the profiling process and may not have been captured. For details, see the following example.
Negative value description: In the preceding figure, the Size column in row 4873 shows a negative value. (The memory allocation size is positive, while the memory deallocation size is negative. If memory is allocated and deallocated within the scope of profiling, then Size takes the allocation value.) However, the Name column cannot identify the operator name, the Allocation columns are all empty, and the Release columns have normal memory deallocation values. This indicates that memory allocation for these operators occurred before the profiling process started, but memory deallocation occurred during the running of the profiling process, resulting in only negative values being collected for memory deallocation. Furthermore, the operator name can be identified only during memory allocation, so it is impossible to identify it during memory deallocation. Additionally, since memory allocation did not occur during the profiling process, the Allocation columns show empty memory allocation values.
Empty value description: In the preceding figure, for operators starting from row 4874, the Release columns are empty while other values are normal. This indicates that while the memory allocation for these operators occurred during the profiling process, the memory deallocation occurred outside this period, resulting in empty entries in the Release columns.