Memory Management Overview

In the Ascend heterogeneous compute architecture, the system consists of hosts and devices. The host and device have their own independent memory. The host memory refers to the host memory (that is, the CPU memory) of the server where the AI processor is located, and the device memory refers to the device memory provided by the AI processor.

There are two things to be done in memory management:

1. Memory access: Runtime provides a set of memory management APIs, enabling you to efficiently and conveniently write memory management code in applications. The host memory and device memory are independent of each other. Runtime provides dedicated APIs to allocate and free host memory and device memory, respectively. For example, aclrtMallocHost and aclrtFreeHost are used to allocate and free host the memory, while aclrtMalloc and aclrtFree are used to allocate and free the device memory.
2. Efficient memory access: To achieve optimal memory access performance, data needs to be stored in the corresponding memory. For example, when an operator is executed on the device, the performance of accessing data on the device is much higher than that of accessing data on the host. Therefore, Runtime provides APIs, such as aclrtMemcpy and aclrtMemcpyAsync, for copying data between the host memory and device memory synchronously or asynchronously. This allows you to better plan data storage and access.