Host Page-Locked Memory Usage

In the CANN programming framework, the host memory can be pageable memory or page-locked memory (also called pinned memory).

• Pageable memory is managed by the OS in a unified manner. You can use traditional APIs such as malloc and mmap to allocate memory and use APIs such as free and munmap to free memory. When the memory pressure is high, the pageable memory is swapped out to the swap space provided by the backup storage. When data in the pageable memory is transferred to the device, the data is first copied to the buffer and then transferred to the device through the Direct Memory Access (DMA) channel.
• Page-locked memory is also called pinned memory. You need to use Runtime APIs, such as aclrtMallocHost and aclrtFreeHost, to allocate and free page-locked memory. For page-locked memory, the mapping between virtual pages and physical pages is fixed and will not be swapped out to the swap space within its lifetime. When data in the page-locked memory is transferred to the device, it is directly transferred through the DMA channel without passing through the buffer, resulting in better transfer performance.
  In Runtime, using page-locked memory has the following advantages:

  • The device can directly access the host memory through DMA without passing through the buffer, providing better transfer performance.
  • The CPU is not required during data transfer, enabling asynchronous data transfer.
  • Asynchronous data transfer allows the transfer process and the computation process to overlap, reducing the overall duration of transfer and computation.

// Initialize resources.
......

// Allocate the page-locked memory.
void *hostPtr = NULL;
aclrtMallocHost(&hostPtr, size);

// Allocate the device memory.
void *devicePtr = NULL;
aclrtMalloc(&devicePtr, size, ACL_MEM_MALLOC_NORMAL_ONLY);

// Initialize the memory.
......

// Asynchronous H2D
aclrtMemcpyAsync(devicePtr, size, hostPtr, size, ACL_MEMCPY_HOST_TO_DEVICE, stream);

// Wait until the asynchronous copy is complete.
aclrtSynchronizeStream(stream);

// Free resources.
if (devicePtr) (void)aclrtFree(devicePtr);
if (hostPtr) (void)aclrtFreeHost(hostPtr);
......

// Allocate the pageable memory.
void *hostPtr = malloc(size);

// Register the memory as page-locked memory (ACL_HOST_REG_PINNED) and map it to the device (ACL_HOST_REG_MAPPED).
aclrtHostRegisterV2(hostPtr, size, ACL_HOST_REG_PINNED|ACL_HOST_REG_MAPPED);

// Obtain the device address.
void *devicePtr = NULL;
aclrtHostGetDevicePointer(hostPtr, &devicePtr, 0);

// The task accesses the corresponding memory through the device address.
...

// Free resources.
aclrtHostUnregister(hostPtr);
free(hostPtr);

If the virtual address (VA) consistency of the host memory is involved:

• The host memory can be registered with the device by calling aclrtHostRegister. The device pointer is obtained based on the mapping of the host pointer for the device to use. By default, the virtual addresses for the same host memory are different for the host and device.
• If the virtual addresses for the host and device need to be the same, you can call aclrtMallocHostWithCfg to allocate page-locked memory and specify the ACL_RT_MEM_ATTR_VA_FLAG attribute in the aclrtMallocConfig parameter. In this way, the virtual address of the device obtained after registration can be the same as that of the host.