Loading a Model

This section describes the API call sequence for network-wide model loading.

API Call Sequence

Two sets of model loading acl APIs are provided for you to choose from based on your programming habits and application scenarios.

Figure 1: You only need to set configuration parameters in APIs for different loading modes (such as loading from a file or from the memory). This has a wider application scope, but multiple APIs need to be used together to create configuration objects, set attribute values in objects, and load models, respectively.
Figure 2: You need to select different APIs according to different loading modes (such as loading from a file or from the memory). The operation is relatively simple, but you need to remember the loading APIs of various modes.
Figure 1 Model loading workflow (setting parameters in the model loading API)

Figure 2 Model loading workflow (using different model loading APIs)

The key APIs are described as follows:

Before loading a model, build offline model adapted to the Ascend AI Processors (*.om file). For details about the build method, see Build the models.
If the memory is managed by the user, call the aclmdlQuerySize API to query the working memory and weight memory required for model running to prevent memory waste.
If the shape of the model input data is uncertain, aclmdlQuerySize cannot be called to query the memory size. As a result, the memory cannot be managed by the user during model loading. Therefore, you need to select a model loading API (for example, aclmdlLoadFromFile) that manages the memory.

During model building, if the graph building API is called to build your own network and the model data is stored in the memory without generating the .om offline model file, the memory size cannot be queried by calling aclmdlQuerySize. For details about the graph construction API, see Graph Development.
A model can be loaded using the following APIs. A model ID is returned after the model is successfully loaded.
- When the aclmdlSetConfigOpt and aclmdlLoadWithConfig APIs are used, the attributes in the configuration object are used to determine whether the model is loaded from the file or memory and whether the memory is managed by the system or user.
- When the following APIs are used, the caller can determine whether to load the model from a file or from memory and whether the memory is managed by the system or the user:
  - aclmdlLoadFromFile: loads offline model data from a file. The memory is managed by the system.
  - aclmdlLoadFromMem: loads offline model data from the memory. The memory for model running is managed by the system.
  - aclmdlLoadFromFileWithMem: loads offline model data from a file. The memory for running the model is managed by the user. The memory includes the working memory and weight memory. The working memory is used to store temporary data during model execution, and the weight memory is used to store weight data.
  - aclmdlLoadFromMemWithMem: loads offline model data from memory. The memory (including working memory and weight memory) is managed by the user.

Sample Code

The following describes how to load a model from a file and manage the memory by the user. After the model is loaded successfully, the ID of the model is returned, which will be used in Running a Model.

The following is a code snippet of key steps only, which is not ready to be built or run. Following the API calls, add exception handling branches and specify log printing of error and information levels.

You can click resnet50_imagenet_classification to obtain the sample.

      
           // 1. Initialize variables.
// The two dots (..) indicate a path relative to the directory of the executable file.
// For example, if the executable file is stored in the out directory, the two dots (..) point to the parent directory of the out directory.
const char* omModelPath = "../model/resnet50.om";
// ......

// 2. Obtain the weight memory size and workspace size required for model execution.
aclError ret = aclmdlQuerySize(omModelPath, &modelMemSize_, &modelWeightSize_);

// 3. Allocate workspace on the device for model execution.
ret = aclrtMalloc(&modelMemPtr_, modelMemSize_, ACL_MEM_MALLOC_HUGE_FIRST);

// 4. Allocate weight memory on the device for model execution.
ret = aclrtMalloc(&modelWeightPtr_, modelWeightSize_, ACL_MEM_MALLOC_HUGE_FIRST);

// 5. Load your offline model. The memory (including the weight memory and workspace) is managed by the user.
// The model is successfully loaded, and the model ID is returned.
ret = aclmdlLoadFromFileWithMem(omModelPath, &modelId_, modelMemPtr_, modelMemSize_, modelWeightPtr_, modelWeightSize_);

// ......

Parent topic: Model Inference with Static-Shape Inputs