Overview

The following table lists the precautions for using the memory of each product model.

Model	Precautions
Atlas 200/300/500 Inference Product	Note that to prevent the huge page memory from being insufficient, the system has reserved a huge page memory about 1940 MB (free OS memory on the device x 25%) in advance. Users should preferentially use this reserved memory. The free OS memory on the device refers to the total memory queried by running the free command. In Ascend EP mode, after the reserved hugepage memory is used up, the system dynamically expands or reduces the hugepage memory based on the memory on the device.
Atlas Training Series Product	In the Ascend EP mode, when an inference or training job is executed for the first time after the device is started, the AI CPU operators are migrated from the host to the device and cached on the device to improve performance. Therefore, some device memory (100 MB to 200 MB, varying depending on the Ascend AI Processor) is occupied. If the device is restarted, the AI CPU operator buffer is released.
All models	Media data processing has higher requirements on the memory for storing the input and output data. Therefore, the dedicated memory allocation APIs need to be called. For details, see the restrictions in Media Data Processing V1 and Media Data Processing V2. The memory allocated by calling 1 can meet the requirements of media data processing and can be used in other tasks. For example, the output of media data processing can be used as the input of model inference to implement memory reuse. Because the address space accessed by media data processing is limited, you are advised to call the APIs under Memory Management (such as aclrtMalloc and aclrtMallocHost) to allocate memory for other functions (such as model loading) to ensure sufficient memory during media data processing.

Model

Precautions

Atlas 200/300/500 Inference Product

Note that to prevent the huge page memory from being insufficient, the system has reserved a huge page memory about 1940 MB (free OS memory on the device x 25%) in advance. Users should preferentially use this reserved memory. The free OS memory on the device refers to the total memory queried by running the free command.

In Ascend EP mode, after the reserved hugepage memory is used up, the system dynamically expands or reduces the hugepage memory based on the memory on the device.

Atlas Training Series Product

In the Ascend EP mode, when an inference or training job is executed for the first time after the device is started, the AI CPU operators are migrated from the host to the device and cached on the device to improve performance. Therefore, some device memory (100 MB to 200 MB, varying depending on the Ascend AI Processor) is occupied. If the device is restarted, the AI CPU operator buffer is released.

All models

Media data processing has higher requirements on the memory for storing the input and output data. Therefore, the dedicated memory allocation APIs need to be called. For details, see the restrictions in Media Data Processing V1 and Media Data Processing V2.
The memory allocated by calling 1 can meet the requirements of media data processing and can be used in other tasks. For example, the output of media data processing can be used as the input of model inference to implement memory reuse.
Because the address space accessed by media data processing is limited, you are advised to call the APIs under Memory Management (such as aclrtMalloc and aclrtMallocHost) to allocate memory for other functions (such as model loading) to ensure sufficient memory during media data processing.

Parent topic: Memory Management