GetCompiledModel

Applicability

Header File/Library File

• Header file: #include <ge/ge_api_v2.h>
• Library file: libge_runner_v2.so

Function Usage

Obtains the serialized model after graph compilation.

Before calling this API, call CompileGraph. After obtaining the serialized model data ModelBufferData from the memory buffer by calling this API, you can perform the follow-up operations in either of the following method:

• Method 1: (Recommended) Generate an OM offline model.
  1. Online compilation: Obtain the serialized model after compilation by calling GetCompiledModel.
  2. Call aclgrphSaveModel to save the serialized model as an OM offline model.

     When calling GetCompiledModel and aclgrphSaveModel, ensure that the CANN software package versions are consistent. The APIs perform no verification.

  3. Call aclmdlLoadFromFile to load the model from a file and then call aclmdlExecute to perform inference.
• Method 2: Do not generate an OM offline model.
  1. Online compilation: Obtain the serialized model after compilation by calling GetCompiledModel.
  2. Call aclmdlLoadFromMem to load the model from the memory and then call aclmdlExecute to perform inference.

Method 1 is recommended. After an OM offline model is generated, the CANN package versions of the GetCompiledModel and aclmdlLoadFromFile APIs can be different. If method 2 is used, the CANN package versions of the GetCompiledModel and aclmdlLoadFromMem APIs must be the same because version compatibility is not guaranteed for the serialized model data ModelBufferData in the memory buffer. Otherwise, undefined behavior may occur.

For details about the APIs, see Model Inference in Application Development (C&C++).

Prototype

1

Status GetCompiledModel(uint32_t graph_id, ModelBufferData &model_buffer)

Parameters

1
2
3
4
5

struct ModelBufferData
{
  std::shared_ptr<uint8_t> data = nullptr;
  uint64_t length;
};

Returns

Restrictions

• The fusion of variables and conversion operators requires graph recompilation. The serialized model obtained by calling GetCompiledModel cannot be compiled again. As such, if a graph contains a variable, set the ge.exec.variable_acc parameter in Options to False to disable the fusion of variables and conversion operators.
• After the external weight function (ge.externalWeight in Options) is enabled, note the following for weight file management:
  • Default behavior: After the external weight function is enabled, the system writes the weight file to a temporary directory by default. (For details about the directory path, see the description about ge.externalWeight.) Note that if the GeSession object is destructed, the temporary directory and all files inside will be automatically deleted.
  • To prevent the weight file from being lost after the GeSession object is destructed, you can perform any of the following to save the weight file persistently:
    • Specify a dedicated directory: You are advised to use the ge.externalWeightDir parameter to specify a dedicated, persistent path for storing the weight file during initialization.
    • Back up the file in advance: Before the GeSession object is destructed, use an application to proactively copy the weight file in the temporary directory to another specified secure directory.
    • Save an offline model: Before the GeSession object is destructed, call aclgrphSaveModel to serialize the compiled model (including the weight) and save it as an independent OM offline model file.

    The preceding methods ensure that the lifecycle of the external weight file is properly managed so that it will not be lost due to the destruction of the GeSession object.