GetCompiledModel

Applicability

Header File/Library File

• Header file: #include <ge/ge_api_v2.h>
• Library file: libge_runner_v2.so

Function Usage

Obtains the serialized model after graph compilation.

Before calling this API, call CompileGraph. After obtaining the serialized model data ModelBufferData from the memory buffer by calling this API, you can perform either of the following operations:

• Usage 1: (Recommended) Generating an OM offline model.
  1. Online compilation: Obtain the serialized model after compilation by calling GetCompiledModel.
  2. Call aclgrphSaveModel to save the serialized model as an OM offline model.

     When calling GetCompiledModel and aclgrphSaveModel, ensure that the CANN software package versions are consistent. No verification is performed in the APIs.

  3. Call acl to load the model API aclmdlLoadFromFile from a file, and then call aclmdlExecute to perform inference.
• Usage 2: Not generating an OM offline model.
  1. Online compilation: Obtain the serialized model after compilation by calling GetCompiledModel.
  2. Call acl to load the model API aclmdlLoadFromMem from the memory, and then call aclmdlExecute to perform inference.

Usage 1 is recommended. After an OM offline model is generated, the CANN package versions of the GetCompiledModel and acl APIs can be different. However, if usage 2 is used, the CANN package versions of the GetCompiledModel and acl APIs must be the same because the version compatibility is not guaranteed for the serialized model data ModelBufferData in the memory buffer. Otherwise, undefined behavior may occur.

For details about acl, see "Model Management" in Application Development Guide (C&C++).

Prototype

1

Status GetCompiledModel(uint32_t graph_id, ModelBufferData &model_buffer)

Parameters

1
2
3
4
5

struct ModelBufferData
{
  std::shared_ptr<uint8_t> data = nullptr;
  uint64_t length;
};

Returns

Constraints

• The fusion of variables and conversion operators requires graph recompilation. The serialized model obtained by calling GetCompiledModel cannot be compiled again. As such, if a graph contains a variable, set the ge.exec.variable_acc parameter in Command-Line Options to False to disable the fusion of variables and conversion operators.
• After the external weight function (ge.externalWeight in Command-Line Options) is enabled, note the following for weight file management:
  • Default behavior: After the external weight function is enabled, the system writes the weight file to a temporary directory by default. (For details about the directory, see the description about ge.externalWeight.) Note that if the GeSession object is destructed, the temporary directory and all files within will be automatically deleted.
  • To prevent the weight file from being lost after the GeSession object is destructed, you can perform any of the following to save the weight file persistently:
    • Specify a dedicated directory: You are advised to use the ge.externalWeightDir parameter to specify a dedicated, persistent path for storing the weight file during initialization.
    • Back up the file in advance: Before the GeSession object is destructed, use an application to proactively copy the weight file in the temporary path to another specified secure directory.
    • Save an offline model: Before the GeSession object is destructed, call aclgrphSaveModel to serialize the compiled model (including the weight) and save it as an independent OM offline model file.

    The preceding methods ensure that the lifecycle of the external weight file can be effectively managed so that it will not be lost due to the destruction of the GeSession object.