Precautions for Memory Usage

The following lists the precautions for memory usage of each product model.

• Media data processing has higher requirements on the memory for storing the input and output data. Therefore, dedicated memory allocation APIs are required, for example, hi_mpi_dvpp_malloc.
• The memory allocated by the dedicated memory allocation APIs can be used for media data processing and other tasks. For example, the output of media data processing can be used as the input of model inference to implement memory reuse and reduce memory copy.
• The address space for media data processing is limited. To ensure sufficient memory for media data processing, you are advised to allocate memory for other functions (for example, model loading) through APIs described in Memory Management, such as aclrtMalloc and aclrtMallocHost.
• For the Ascend 950PR / Ascend 950DT , Atlas A3 training products / Atlas A3 inference products , and Atlas A2 training products / Atlas A2 inference products , if huge page memory needs to be allocated on the device, note that in the current version, huge page memory has been reserved in the system in case of insufficiency. Before using huge page memory, you can call aclrtGetMemInfo to query the idle huge page memory.
• For the Ascend 950PR / Ascend 950DT , Atlas A3 training products / Atlas A3 inference products , or Atlas A2 training products / Atlas A2 inference products in Ascend EP form, when an inference or training task is executed for the first time after the device is started, the system package is loaded to the device. Therefore, some device memory (about 300 MB, varying depending on the AI processor) is occupied. This part of memory is freed only when the device is restarted.
• For the Atlas 200I/500 A2 inference products , Atlas inference products , or Atlas training products in Ascend EP form, when an inference task is executed for the first time after the device is started, the AI CPU operators are migrated from the host to the device and cached on the device to improve performance. Therefore, some device memory (100 MB to 200 MB, varying depending on the AI processor) is occupied. The AI CPU operator cache is freed only when the device is restarted.