aclrtMemMallocPolicy

typedef enum aclrtMemMallocPolicy {
    ACL_MEM_MALLOC_HUGE_FIRST,
    ACL_MEM_MALLOC_HUGE_ONLY,
    ACL_MEM_MALLOC_NORMAL_ONLY,
    ACL_MEM_MALLOC_HUGE_FIRST_P2P,
    ACL_MEM_MALLOC_HUGE_ONLY_P2P,
    ACL_MEM_MALLOC_NORMAL_ONLY_P2P,
    ACL_MEM_MALLOC_HUGE1G_ONLY, 
    ACL_MEM_MALLOC_HUGE1G_ONLY_P2P,
    ACL_MEM_TYPE_LOW_BAND_WIDTH   = 0x0100U,
    ACL_MEM_TYPE_HIGH_BAND_WIDTH  = 0x1000U,
    ACL_MEM_ACCESS_USER_SPACE_READONLY = 0x100000U,
} aclrtMemMallocPolicy;

Both a single enumeration and a bitwise OR operation for multiple enumerations are supported.

Configure a single enumeration:
- If ACL_MEM_TYPE_LOW_BAND_WIDTH or ACL_MEM_TYPE_HIGH_BAND_WIDTH is configured, the system uses ACL_MEM_MALLOC_HUGE_FIRST by default and preferentially allocates huge pages.
- If a value other than ACL_MEM_TYPE_LOW_BAND_WIDTH and ACL_MEM_TYPE_HIGH_BAND_WIDTH is configured, the system allocates memory from either the high-bandwidth or low-bandwidth physical memory, depending on the hardware situation.
Configure a bitwise OR operation for multiple enumerations:
You can select an item from ACL_MEM_MALLOC_HUGE_FIRST, ACL_MEM_MALLOC_HUGE_ONLY, and ACL_MEM_MALLOC_NORMAL_ONLY, select an item from ACL_MEM_TYPE_LOW_BAND_WIDTH and ACL_MEM_TYPE_HIGH_BAND_WIDTH, and perform a bitwise OR operation on the two items, for example, ACL_MEM_MALLOC_HUGE_FIRST | ACL_MEM_TYPE_HIGH_BAND_WIDTH.

The enumerations are described as follows:

ACL_MEM_MALLOC_HUGE_FIRST
Allocates huge page memory with a granularity of 2 MB. If the allocated memory is not a multiple of 2 MB, it is rounded up to the nearest multiple of 2 MB.

When the allocated memory is less than or equal to 1 MB, only normal page memory is allocated even under this memory allocation rule. When the allocated memory is greater than 1 MB, huge page memory is allocated first. If the huge page memory is insufficient, normal page memory is allocated.
ACL_MEM_MALLOC_HUGE_ONLY
Allocates huge page memory with a granularity of 2 MB. If the allocated memory is not a multiple of 2 MB, it is rounded up to the nearest multiple of 2 MB.

If this item is configured, only huge page memory is allocated. If the huge page memory is insufficient, an error is returned.
ACL_MEM_MALLOC_NORMAL_ONLY
Allocates only normal page memory. If the normal page memory is insufficient, an error is returned.
ACL_MEM_MALLOC_HUGE_FIRST_P2P
Allocates huge page memory with a granularity of 2 MB only in the scenario of memory copy between two devices. If the allocated memory is not a multiple of 2 MB, it is rounded up to the nearest multiple of 2 MB.

If this item is configured, huge page memory is allocated first. If the huge page memory is insufficient, normal page memory is allocated.

For the Atlas 200I/500 A2 inference products, this item is not supported.

For the Atlas inference products, if collective communication is involved, the communicator must be initialized before any other operations that require device memory allocation. Otherwise, the initialization may fail due to insufficient P2P memory.
ACL_MEM_MALLOC_HUGE_ONLY_P2P
Allocates huge page memory with a granularity of 2 MB only in the scenario of memory copy between two devices. If the allocated memory is not a multiple of 2 MB, it is rounded up to the nearest multiple of 2 MB.

If this item is configured, only huge page memory is allocated. If the huge page memory is insufficient, an error is returned.

For the Atlas 200I/500 A2 inference products, this item is not supported.

For the Atlas inference products, if collective communication is involved, the communicator must be initialized before any other operations that require device memory allocation. Otherwise, the initialization may fail due to insufficient P2P memory.
ACL_MEM_MALLOC_NORMAL_ONLY_P2P
Allocates only normal page memory in the scenario of memory copy between two devices.

For the Atlas 200I/500 A2 inference products, this item is not supported.

For the Atlas inference products, if collective communication is involved, the communicator must be initialized before any other operations that require device memory allocation. Otherwise, the initialization may fail due to insufficient P2P memory.
ACL_MEM_MALLOC_HUGE1G_ONLY
Allocates huge page memory with a granularity of 1 GB. If the allocated memory is not a multiple of 1 GB, it is rounded up to the nearest multiple of 1 GB. For example, if 1.9 GB memory is requested, 2 GB memory is actually allocated according to the round-up rule.

If this item is configured, only 1 GB huge page memory is allocated. If the 1 GB huge page memory is insufficient, an error is returned. Because 1 GB huge page memory is limited, you can use the ACL_MEM_MALLOC_HUGE_FIRST item to allocate huge page memory.

To allocate 1 GB of huge page memory, ACL_MEM_MALLOC_HUGE_ONLY (with a 2 MB allocation granularity) requires 512 (= 1024/2) page tables, while ACL_MEM_MALLOC_HUGE1G_ONLY (with a 1 GB granularity) requires only one page table. The latter effectively reduces the number of page tables and expands the address range of the translation lookaside buffer (TLB), thereby improving the discrete access performance. TLB is a hardware module in the AI processor used for caching. It stores the mapping between the recently used virtual addresses and physical addresses.

For the Atlas 200I/500 A2 inference products, Atlas inference products, and Atlas training products, this item is not supported.
ACL_MEM_MALLOC_HUGE1G_ONLY_P2P
Allocates huge page memory with a granularity of 1 GB only in the scenario of memory copy between two devices. If the allocated memory is not a multiple of 1 GB, it is rounded up to the nearest multiple of 1 GB. For example, if 1.9 GB memory is requested, 2 GB memory is actually allocated according to the round-up rule.

If this item is configured, only 1 GB huge page memory is allocated. If the 1 GB huge page memory is insufficient, an error is returned. Because 1 GB huge page memory is limited, you can use the ACL_MEM_MALLOC_HUGE_FIRST_P2P item to allocate huge page memory.

To allocate 1 GB of huge page memory, ACL_MEM_MALLOC_HUGE_ONLY_P2P (with a 2 MB allocation granularity) requires 512 (= 1024/2) page tables, while ACL_MEM_MALLOC_HUGE1G_ONLY_P2P (with a 1 GB granularity) requires only one page table. The latter effectively reduces the number of page tables and expands the address range of the TLB, thereby improving the discrete access performance. TLB is a hardware module in the AI processor used for caching. It stores the mapping between the recently used virtual addresses and physical addresses.

For the Atlas 200I/500 A2 inference products, Atlas inference products, and Atlas training products, this item is not supported.
ACL_MEM_TYPE_LOW_BAND_WIDTH
Allocates memory from the low-bandwidth physical memory.

This item is invalid. By default, the system selects the allocation rule based on hardware-supported memory types.
ACL_MEM_TYPE_HIGH_BAND_WIDTH
Allocates memory from the high-bandwidth physical memory.

This item is invalid. By default, the system selects the allocation rule based on hardware-supported memory types.
ACL_MEM_ACCESS_USER_SPACE_READONLY
Specifies that the allocated memory is read-only in user mode. Memory modification in user mode would fail.

Parent topic: Data Types and Operation APIs