aclmdlExecuteV2

Applicability

Product	Supported
Atlas A3 training products/Atlas A3 inference products	√
Atlas A2 training products/Atlas A2 inference products	√
Atlas 200I/500 A2 inference products	√
Atlas inference products	√
Atlas training products	√

Description

Performs model inference based on the attributes configured by the aclmdlSetExecConfigOpt API until the inference result is returned. This API is enhanced based on aclmdlExecute. It allows you to control configuration parameters during model inference.

This API must be used together with other APIs to implement model execution. The API call sequence is as follows:

Call aclmdlCreateExecConfigHandle to create a configuration object for model execution.
Call aclmdlSetExecConfigOpt multiple times to set the value of each attribute in the configuration object.
Call aclmdlExecuteV2 to specify the configuration information required for model execution and execute the model.
After the model is successfully executed, call aclmdlDestroyExecConfigHandle to destroy the configuration object.

Prototype

aclError aclmdlExecuteV2(uint32_t modelId, const aclmdlDataset *input, aclmdlDataset *output, aclrtStream stream, const aclmdlExecConfigHandle *handle)

Parameters

Parameter

Input/Output

Description

modelId

Input

Model ID for inference.

After a model loading API (such as aclmdlLoadFromFile and aclmdlLoadFromMem) is successfully called, a model ID is returned. The ID is used as the input of this API.

input

Input

Pointer to the input data for model inference.

output

Output

Pointer to the output data for model inference.

When calling aclCreateDataBuffer to create the aclDataBuffer type for storing output data of the corresponding index, you can pass nullptr to the data parameter and set size to 0 to create an empty aclDataBuffer type. During model execution, the system automatically calculates and allocates the index output buffer. This method saves memory. However, you need to free the memory and reset the aclDataBuffer after using the data. In addition, memory copy is involved when the system allocates memory, which may cause performance loss.

The sample code for freeing the memory and resetting the aclDataBuffer is as follows:

aclDataBuffer *dataBuffer = aclmdlGetDatasetBuffer(output, 0); // Obtain the corresponding data buffer based on the index.
void *data = aclGetDataBufferAddr(dataBuffer);  // Obtain the device pointer to data.
aclrtFree(data ); // Free the device memory.
aclUpdateDataBuffer(dataBuffer, nullptr, 0); // Reset the content in the data buffer for next inference.

stream

Input

Stream.

If NULL is passed, the attribute value configured by the aclmdlSetExecConfigOpt API does not take effect.

handle

Input

Pointer to the configuration object executed by the model. The value must be the same as that of handle in aclmdlSetExecConfigOpt.

Returns

0 on success; else, failure. For details, see aclError.

Restrictions

The restrictions are the same as those of aclmdlExecute.

Parent topic: Model Execution