Documentation

Overview

  • If media data processing is involved, pay attention to the following points when using the memory:
    1. Media data processing has higher requirements on the memory for storing the input and output data (for example, the start address of the memory must be 128-byte aligned). Therefore, the following dedicated memory allocation APIs are required:
    2. The memory allocated by the preceding APIs can be used for media data processing and other tasks. For example, the output of media data processing can be used as the input of model inference to implement memory overcommitment and reduce memory copy.
    3. Because the address space accessed by media data processing is limited, you are advised to call acl.rt.malloc, acl.rt.malloc_host, or acl.rt.malloc_cached described in section "Memory Management" to allocate memory for other functions (for example, model loading) to ensure sufficient memory during media data processing.
  • For the Atlas 200/300/500 Inference Product, note that to prevent the huge page memory from being insufficient, the system has reserved a huge page memory about 1940 MB (Free OS memory on the device x 25%) in advance. You should preferentially use this reserved memory. The free OS memory on the device refers to the total memory queried by running the free command.
    • In standard form, after the reserved huge page memory is used, the system dynamically expands or reduces the huge page memory based on the device memory.
  • In the Ascend EP mode, when an inference or training task is executed for the first time after the device is started, the AI CPU operators are migrated from the host to the device and cached on the device to improve performance. Therefore, some device memory (100 MB to 200 MB, varying depending on the chip) is occupied. If the device is restarted, the AI CPU operator buffer is released.

    This restriction applies to the following products:

    • Atlas Training Series Product
Parent topic: Memory Management