aclmdlLoadFromMemWithMem

Description

Loads offline model data from memory. The model workspace is managed by the user.

Restrictions

The operations of loading, executing, and unloading a model must be performed in the same context. For details about how to create a context, see aclrtSetDevice or aclrtCreateContext. Before loading the model file, check whether the memory space is sufficient based on the size of the model file. If the memory space is insufficient, the application will be abnormal.

Prototype

aclError aclmdlLoadFromMemWithMem(const void *model, size_t modelSize, uint32_t *modelId, void *workPtr, size_t workSize, void *weightPtr, size_t weightSize)

Parameters

Parameter	Input/Output	Description
model	Input	Pointer to the address of the memory that stores the model. Allocate host memory or device memory based on where the app runs. For details about the memory allocation API, see Memory Management.
modelSize	Input	Model size in bytes.
modelId	Output	Pointer to the model ID. Returns the model ID after the model is successfully loaded. The model ID is used for model identification in subsequent operations.
workPtr	Input	Pointer to the address of the workspace (for storing temporary data during model execution) required by the model on the device. The workspace is managed by the user and cannot be freed during model execution. If a null pointer is passed to workPtr, the workspace is managed by the system. NOTE: In the event where the memory is managed by the user, if multiple models are executed in serial, the models can share a workspace. However, you need to guarantee the serial execution sequence of the models and the workspace size (the same as the total size of the workspaces needed by all the models). Refer to the following description to ensure serial execution: For synchronous model execution, add a lock to ensure that tasks are executed in serial. For asynchronous model execution, use a single stream to ensure that tasks are executed in serial.
workSize	Input	Workspace size required for model execution, in bytes. Invalid when workPtr is a null pointer.
weightPtr	Input	Pointer to the address of the model weight memory (for storing weight data) on the device. The memory is managed by the user and cannot be freed during model execution. If a null pointer is passed, the memory is managed by the system. NOTE: In the multithreaded scenario where the weight memory is managed by the user, if a model is loaded once in each thread, weightPtr can be shared because the weight memory is read-only at inference time. Note that weightPtr cannot be freed when the sharing is in progress.
weightSize	Input	Weight memory size required for model execution, in bytes. Invalid when weightPtr is a null pointer.

Returns

The value 0 indicates success, and other values indicate failure. For details, see aclError.

Related APIs

AscendCL also provides the aclmdlSetConfigOpt and aclmdlLoadWithConfig APIs for model loading. The caller needs to set the attributes in the configuration object passed to the API call to decide how the model will be loaded and who will manage the memory.