Function: execute_async
Function Usage
Executes model inference. It is an asynchronous interface.
Prototype
- C Prototype
1aclError aclmdlExecuteAsync(uint32_t modelId, const aclmdlDataset *input, aclmdlDataset *output, aclrtStream stream)
- Python Function
1ret = acl.mdl.execute_async(model_id, input, output, stream)
Parameters
|
Parameter |
Description |
|---|---|
|
model_id |
Int, ID of the model to be executed for inference. You can obtain the model ID after the model is successfully loaded by calling the following APIs: |
|
input |
Int, pointer address of the input data for model inference. For details, see aclmdlDataset. |
|
output |
Int, pointer address of the output data for model inference. For details, see aclmdlDataset. |
|
stream |
Int, pointer address of the created stream. To specify a new stream, you can create and obtain the pointer address of the stream by calling acl.rt.create_stream. |
Returns
|
Return Value |
Description |
|---|---|
|
ret |
Int, error code. 0 indicates success, and other values indicate failure. |
Restrictions
- This API is asynchronous. The API call delivers a task rather than executes a task. After this API is called, call the synchronization API (for example, acl.rt.synchronize_stream) to ensure that the task is complete.
- For models with the same model_id, acl.mdl.execute_async cannot be called to perform model inference in the multi-stream concurrency scenario.
- If the same model_id is shared by multiple threads, locks must be added between user threads to ensure that operations of refreshing the input and output memory and executing inference are performed continuously. For example:
// API call sequence of thread A: lock(handle1) -> acl.rt.memcpy_async(stream1) (Refresh the input and output memory) - > acl.mdl.execute_async(modelId1,stream1) (Execute inference) - > unlock(handle1) // API call sequence of thread B: lock(handle1) -> acl.rt.memcpy_async(stream1) (Refresh the input and output memory) - > acl.mdl.execute_async(modelId1,stream1) (Execute inference) - > unlock(handle1)
- Model loading, execution, and unloading must be performed in the same context. For details about how to create a context, see acl.rt.create_context.
- You can use the following APIs to allocate memory for storing model input and output data: acl.rt.malloc, acl.rt.malloc_host, acl.rt.malloc_cached, acl.media.dvpp_malloc, or acl.himpi.dvpp_malloc.
acl.media.dvpp_malloc and acl.himpi.dvpp_malloc are dedicated memory allocation APIs for media data processing. To reduce copy, the output of media data processing is used as the input of model inference to implement memory overcommitment.
- The address space accessed by media data processing is limited. To ensure sufficient memory during media data processing, you are advised to call other APIs (such as the acl.rt.malloc API) to allocate memory for other functions (such as model loading) except media data processing.