Memory Management

EXEC_DISABLE_REUSED_MEMORY

Memory reuse switch. The corresponding options parameter is ge.exec.disableReuseMemory.

Memory reuse refers to the practice of repeatedly utilizing non-conflicting memory based on its lifecycle and size, thereby reducing network memory consumption.

Arguments:

• 0 (default): enabled
• 1: disabled If the network model is large, disabling memory reuse will cause the device memory not to be reused during subsequent inference, resulting in insufficient memory.

Configuration example:

{ge::ir_option::EXEC_DISABLE_REUSED_MEMORY, "0"}

Applicable products:

All processors

EXTERNAL_WEIGHT

Whether to externalize the weights of the Const/Constant nodes on the original network and convert the node type to FileConstant when the OM model file is generated. The corresponding options parameter is ge.externalWeight.

In the offline scenario, if the model weight is large and the environment has restrictions on the OM offline model file size, you are advised to enable the external weight and save the weight separately to reduce the OM file size.

Arguments:

• 0 (default): The weights are not externalized and are directly saved in the OM offline model file.
• 1: The weights are externalized but are not normalized. The weight files of all Const/Constant nodes on the network are flushed to disk, and the node type is converted to FileConstant. The weight files are saved in the weight directory at the same level as the OM file. Weights of different nodes are stored in different files, which are named in the format of weight_<hash value>.
• 2: The weights are externalized and are normalized. The weight files of all Const/Constant nodes on the network are flushed to disk, and the node type is converted to FileConstant. The weight files are saved in the weight directory at the same level as the OM file. All weights are stored in the same file, which is named in the format of <original root graph name>_weight_combined.

Configuration example:

{ge::ir_option::EXTERNAL_WEIGHT, "1"}

Restrictions:

• In the external weight scenario, when developing inference applications and loading models:
  • Use the aclgrphSaveModel API to save the OM model.
    • If aclmdlLoadFromFile is used to load a model, the weight file must be stored in the weight directory at the same level as the OM file.
    • If aclmdlSetConfigOpt and aclmdlLoadWithConfig are used to load a model, there is no requirement on the external weight directory. When the model is loaded later, use aclmdlLoadWithConfig to specify the external weight directory.
  • In the weight update scenario, use aclgrphBundleSaveModel to save the OM model.

    Only aclmdlBundleLoadFromFile can be used to load a model, and the weight file must be stored in the weight directory at the same level as the OM file.

  For details about the APIs, see Model Loading and Unloading.

Applicable products:

All processors