Precautions for Memory Usage

The following table lists the precautions for memory usage of each product model.

Model	Precautions
All models	Media data processing has higher requirements on the memory for storing the input and output data. Therefore, dedicated memory allocation APIs are required, for example, hi_mpi_dvpp_malloc. The memory allocated by the dedicated memory allocation APIs can be used for media data processing and other tasks. For example, the output of media data processing can be used as the input of model inference to implement memory reuse and reduce memory copy. The address space accessed by media data processing is limited. You are advised to call an API described in Memory Management to allocate memory for other functions (for example, model loading). For example, call aclrtMalloc or aclrtMallocHost to ensure sufficient memory during media data processing.
Atlas 350 Accelerator Card Atlas A3 training product / Atlas A3 inference product Atlas A2 training product / Atlas A2 inference product	If huge page memory needs to be allocated on the device, note that in the current version, huge page memory has been reserved in the system in case of insufficiency. Before using huge page memory, you can call aclrtGetMemInfo to query the idle huge page memory. In the Ascend EP form, when an inference or training job is executed for the first time after the device is started, the system package is loaded to the device. Therefore, some device memory (about 300 MB, varying depending on the AI processor) is occupied. This part of memory is freed only when the device is restarted.
Atlas 200I/500 A2 inference product Atlas inference product Atlas training product	In the Ascend EP form, when an inference job is executed for the first time after the device is started, the AI CPU operators are migrated from the host to the device and cached on the device to improve performance. Therefore, some device memory (100 MB to 200 MB, varying depending on the AI processor) is occupied. The AI CPU operator cache is freed only when the device is restarted.

Model

Precautions

All models

Media data processing has higher requirements on the memory for storing the input and output data. Therefore, dedicated memory allocation APIs are required, for example, hi_mpi_dvpp_malloc.
The memory allocated by the dedicated memory allocation APIs can be used for media data processing and other tasks. For example, the output of media data processing can be used as the input of model inference to implement memory reuse and reduce memory copy.
The address space accessed by media data processing is limited. You are advised to call an API described in Memory Management to allocate memory for other functions (for example, model loading). For example, call aclrtMalloc or aclrtMallocHost to ensure sufficient memory during media data processing.

Atlas 350 Accelerator Card

Atlas A3 training product / Atlas A3 inference product

Atlas A2 training product / Atlas A2 inference product

If huge page memory needs to be allocated on the device, note that in the current version, huge page memory has been reserved in the system in case of insufficiency. Before using huge page memory, you can call aclrtGetMemInfo to query the idle huge page memory.
In the Ascend EP form, when an inference or training job is executed for the first time after the device is started, the system package is loaded to the device. Therefore, some device memory (about 300 MB, varying depending on the AI processor) is occupied. This part of memory is freed only when the device is restarted.

Atlas 200I/500 A2 inference product

Atlas inference product

Atlas training product

In the Ascend EP form, when an inference job is executed for the first time after the device is started, the AI CPU operators are migrated from the host to the device and cached on the device to improve performance. Therefore, some device memory (100 MB to 200 MB, varying depending on the AI processor) is occupied. The AI CPU operator cache is freed only when the device is restarted.

Parent topic: Memory Management