aclmdlExecute

Applicable Products

Product	Supported
Ascend 950PR / Ascend 950DT	√
Atlas A3 training products / Atlas A3 inference products	√
Atlas A2 training products / Atlas A2 inference products	√
Atlas 200I/500 A2 inference products	√
Atlas inference products	√
Atlas training products	√

Function Usage

Executes model inference until the result is returned.

The operations of loading, executing, and unloading a model must be performed in the same context. For details about how to create a context, see aclrtCreateContext.

Prototype

      
           aclError aclmdlExecute(uint32_t modelId, const aclmdlDataset *input, aclmdlDataset *output)

Parameters

Parameter

Input/Output

Description

modelId

Input

Inference model ID.

After a model loading API (such as aclmdlLoadFromFile or aclmdlLoadFromMem) is successfully called, the model ID is returned. This ID is used as the input of this API.

input

Input

Pointer to the input data for model inference. For the type definition, see aclmdlDataset.

output

Output

Pointer to the output data for model inference. For the type definition, see aclmdlDataset.

When calling aclCreateDataBuffer to create an aclDataBuffer type for storing the output data of the corresponding index, you can pass nullptr to the data parameter and set size to 0 to create an empty aclDataBuffer type. During model execution, the system automatically calculates and allocates the index output memory. This method saves memory. However, you need to free the memory and reset the aclDataBuffer after using the data. In addition, memory copy is involved when the system allocates memory, which may cause performance loss.

The sample code for freeing the memory and resetting the aclDataBuffer is as follows:

            
                 aclDataBuffer *dataBuffer = aclmdlGetDatasetBuffer(output, 0); // Obtain the corresponding data buffer based on the index.
void *data = aclGetDataBufferAddr(dataBuffer);  // Obtain the device pointer to data.
aclrtFree(data ); // Free the device memory.
aclUpdateDataBuffer(dataBuffer, nullptr, 0); // Reset the content in the data buffer for next inference.

Returns

0 on success; else, failure. For details, see aclError.

Restrictions

If the same modelId is shared by multiple threads due to service requirements, locks must be added between user threads to ensure that operations of refreshing the input and output memory and executing inference are performed continuously. For example:

// API call sequence of thread A:
lock(handle1) -> aclrtMemcpy (Refresh the input and output memory) -> aclmdlExecute(Execute inference) -> unlock(handle1)

// API call sequence of thread B:
lock(handle1) -> aclrtMemcpy (Refresh the input and output memory) -> aclmdlExecute(Execute inference) -> unlock(handle1)

Device memory is used for storing model input and output data. It can be allocated by calling APIs such as aclrtMalloc and hi_mpi_dvpp_malloc. The hi_mpi_dvpp_malloc API is a dedicated memory allocation API for media data processing. To reduce copying, the output of media data processing is used as the input of model inference, implementing memory reuse. The address space accessible by media data processing is limited. To ensure sufficient memory during media data processing, you are advised to call other APIs (such as aclrtMalloc) to allocate memory for other functions (such as model loading).

Example

For details about the API call sequence and sample code, see Running a Model.

Parent topic: Model Execution