aclmdlLoadFromFileWithMem

Description

Loads offline model data (offline model adapted to the Ascend AI Processor) from a file. The model workspace is managed by the user.

Restrictions

The operations of loading, executing, and unloading a model must be performed in the same context. For details about how to create a context, see aclrtSetDevice or aclrtCreateContext.

Prototype

aclError aclmdlLoadFromFileWithMem(const char *modelPath, uint32_t *modelId, void *workPtr, size_t workSize, void *weightPtr, size_t weightSize)

Parameters

Parameter	Input/Output	Description
modelPath	Input	Pointer to the model file path, which contains the file name. The user who runs the app must have enough permission to access the directory. The .om file is an offline model adapted to the Ascend AI Processor. NOTE: The .om file is an offline model adapted to the Ascend AI Processor. For details about how to obtain the .om file, see --mode Parameters in ATC Instructions.
modelId	Output	Pointer to the model ID. Returns the model ID after the model is successfully loaded. The model ID is used for model identification in subsequent operations.
workPtr	Input	Pointer to the address of the workspace (for storing temporary data during model execution) required by the model on the device. The workspace is managed by the user and cannot be freed during model execution. If a null pointer is passed to workPtr, the workspace is managed by the system. NOTE: In the event where the memory is managed by the user, if multiple models are executed in serial, the models can share a workspace. However, you need to guarantee the serial execution sequence of the models and the workspace size (the same as the total size of the workspaces needed by all the models). Refer to the following description to ensure serial execution: For synchronous model execution, add a lock to ensure that tasks are executed in serial. For asynchronous model execution, use a single stream to ensure that tasks are executed in serial.
workSize	Input	Workspace size required for model execution, in bytes. Invalid when workPtr is a null pointer.
weightPtr	Input	Pointer to the address of the model weight memory (for storing weight data) on the device. The memory is managed by the user and cannot be freed during model execution. If a null pointer is passed, the memory is managed by the system. NOTE: In the multithreaded scenario where the weight memory is managed by the user, if a model is loaded once in each thread, weightPtr can be shared because the weight memory is read-only at inference time. Note that weightPtr cannot be freed when the sharing is in progress.
weightSize	Input	Weight memory size required for model execution, in bytes. Invalid when weightPtr is a null pointer.

Returns

The value 0 indicates success, and other values indicate failure. For details, see aclError.

Related APIs

AscendCL also provides the aclmdlSetConfigOpt and aclmdlLoadWithConfig APIs for model loading. The caller needs to set the attributes in the configuration object passed to the API call to decide how the model will be loaded and who will manage the memory.