Precautions for Memory Usage

The following table lists the precautions for memory usage of each product model.

Model	Precautions
All models	Media data processing has higher requirements on the memory for storing the input and output data. Therefore, dedicated memory allocation APIs are required, for example, . The memory allocated by the dedicated memory allocation APIs can be used for media data processing and other tasks. For example, the output of media data processing can be used as the input of model inference to implement memory reuse and reduce memory copy. The address space accessed by media data processing is limited. You are advised to call an API described in Memory Management to allocate memory for other functions (for example, model loading). For example, call aclrtMalloc or aclrtMallocHost) to ensure sufficient memory during media data processing.
Atlas A3 training products/Atlas A3 inference products Atlas A2 training products/Atlas A2 inference products	If huge page memory needs to be allocated on the device, note that in the current version, huge page memory has been reserved in the system in case of insufficiency. Before using huge page memory, you can call aclrtGetMemInfo to query the idle huge page memory. In the Ascend EP form, when an inference or training job is executed for the first time after the device is started, the system package is loaded to the device. Therefore, some device memory (about 300 MB, varying depending on the Ascend AI Processor) is occupied. This part of memory is freed only when the device is restarted.
Atlas 200I/500 A2 inference products Atlas inference products Atlas training products	In the Ascend EP form, when an inference job is executed for the first time after the device is started, the AI CPU operators are migrated from the host to the device and cached on the device to improve performance. Therefore, some device memory (100 MB to 200 MB, varying depending on the Ascend AI Processor) is occupied. The AI CPU operator cache is freed only when the device is restarted.

Model

Precautions

All models

Media data processing has higher requirements on the memory for storing the input and output data. Therefore, dedicated memory allocation APIs are required, for example, .
The memory allocated by the dedicated memory allocation APIs can be used for media data processing and other tasks. For example, the output of media data processing can be used as the input of model inference to implement memory reuse and reduce memory copy.
The address space accessed by media data processing is limited. You are advised to call an API described in Memory Management to allocate memory for other functions (for example, model loading). For example, call aclrtMalloc or aclrtMallocHost) to ensure sufficient memory during media data processing.

Atlas A3 training products/Atlas A3 inference products

Atlas A2 training products/Atlas A2 inference products

If huge page memory needs to be allocated on the device, note that in the current version, huge page memory has been reserved in the system in case of insufficiency. Before using huge page memory, you can call aclrtGetMemInfo to query the idle huge page memory.
In the Ascend EP form, when an inference or training job is executed for the first time after the device is started, the system package is loaded to the device. Therefore, some device memory (about 300 MB, varying depending on the Ascend AI Processor) is occupied. This part of memory is freed only when the device is restarted.

Atlas 200I/500 A2 inference products

Atlas inference products

Atlas training products

In the Ascend EP form, when an inference job is executed for the first time after the device is started, the AI CPU operators are migrated from the host to the device and cached on the device to improve performance. Therefore, some device memory (100 MB to 200 MB, varying depending on the Ascend AI Processor) is occupied. The AI CPU operator cache is freed only when the device is restarted.

Parent topic: Memory Management