aclrtMemAttr

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
typedef enum aclrtMemAttr {
    ACL_DDR_MEM,             // Huge page memory + normal page memory
    ACL_HBM_MEM,             // Huge page memory + normal page memory
    ACL_DDR_MEM_HUGE,        // Huge page memory
    ACL_DDR_MEM_NORMAL,      // Normal page memory
    ACL_HBM_MEM_HUGE,        // Huge page memory. The memory allocation granularity is 2 MB. If the allocated memory is not a multiple of 2 MB, it is rounded up to the nearest multiple of 2 MB.
    ACL_HBM_MEM_NORMAL,      // Normal page memory
    ACL_DDR_MEM_P2P_HUGE,    // Huge page memory for inter-device data copy
    ACL_DDR_MEM_P2P_NORMAL,  // Normal page memory for inter-device data copy
    ACL_HBM_MEM_P2P_HUGE,    // Huge page memory for inter-device data copy. The memory allocation granularity is 2 MB. If the allocated memory is not a multiple of 2 MB, it is rounded up to the nearest multiple of 2 MB.
    ACL_HBM_MEM_P2P_NORMAL,  // Normal page memory for inter-device data copy
    ACL_HBM_MEM_HUGE1G,      // Huge page memory. The memory allocation granularity is 1 GB. If the allocated memory is not a multiple of 1 GB, it is rounded up to the nearest multiple of 1 GB.
    ACL_HBM_MEM_P2P_HUGE1G   // Huge page memory for inter-device data copy. The memory allocation granularity is 1 GB. If the allocated memory is not a multiple of 1 GB, it is rounded up to the nearest multiple of 1 GB.

    /* The preceding options are compatible with earlier versions. You need to select memory attribute options that match the hardware memory (DDR or HBM). */
    /* The following options are automatically adapted by the API based on the underlying hardware memory (DDR or HBM). You do not need to distinguish hardware details. Such options are recommended. */

    ACL_MEM_NORMAL,          // Normal page memory
    ACL_MEM_HUGE,            // Huge page memory. The memory allocation granularity is 2 MB. If the allocated memory is not a multiple of 2 MB, it is rounded up to the nearest multiple of 2 MB.
    ACL_MEM_HUGE1G,          // Huge page memory. The memory allocation granularity is 1 GB. If the allocated memory is not a multiple of 1 GB, it is rounded up to the nearest multiple of 1 GB.
    ACL_MEM_P2P_NORMAL,      // Normal page memory for inter-device data copy
    ACL_MEM_P2P_HUGE,        // Huge page memory for inter-device data copy. The memory allocation granularity is 2 MB. If the allocated memory is not a multiple of 2 MB, it is rounded up to the nearest multiple of 2 MB.
    ACL_MEM_P2P_HUGE1G,      // Huge page memory for inter-device data copy. The memory allocation granularity is 1 GB. If the allocated memory is not a multiple of 1 GB, it is rounded up to the nearest multiple of 1 GB.
} aclrtMemAttr;

To allocate 1 GB of huge page memory, a 2 MB allocation granularity requires 512 (1024/2) page tables, while a 1 GB granularity requires only one page table. The latter effectively reduces the number of page tables and expands the address range of the translation lookaside buffer (TLB), thereby improving the discrete access performance. TLB is a hardware module in the AI processor used for caching. It stores the mapping between the recently used virtual addresses and physical addresses.

HUGE1G-related options are supported only for the Atlas A3 training product/Atlas A3 inference product and Atlas A2 training product/Atlas A2 inference product.

Currently, HUGE1G-related options are not supported for other models.