Asynchronous Invocation

Function Description

By default, Vision SDK adopts synchronous execution. Some APIs support asynchronous execution by allocating AscendStream. For details about the APIs that support the asynchronous mode, see API Reference (C++).

For details about related APIs, see Asynchronous Invocation.

API Call Process

• Customize the AscendStream class to construct the required asynchronous API for inputing the stream instance, input the stream through the API, and execute the stream on the asynchronous API. After an API task is specified for the same stream in serial mode, you can call the Synchronize() API to block application or thread running until all tasks in the stream are complete.
• Asynchronous execution of multiple streams is supported. APIs in each stream are executed in sequence. If asynchronous execution of multiple streams is used, or the asynchronous execution result needs to be input to an API that does not support asynchronous execution, call the Synchronize() API to synchronize streams in a proper position to ensure that the result is correctly returned for subsequent use.
• When processing media data in the asynchronous mode, call Synchronize() to complete the asynchronous task so that the used channel can be returned in time, preventing the resource pool from being used up.

Figure 1 Process of calling APIs in stream asynchronous mode (resizing as an example)

Vision SDK provides the AscendStream class for stream management. The key steps are described as follows:

1. Perform global initialization by calling MxInit().
2. Initialize the stream.

   Pass deviceId to the constructor to specify the device for which the stream is to be created. The supported deviceId vary according to the user environment.

3. Create a stream.

   This operation must be performed together with stream initialization. Before using a stream, call the member function of CreateAscendStream() to create it.

4. Call the asynchronous API.
   • A specified stream can be passed through an asynchronous API. APIs in the same stream are executed in sequence.
   • Specify the parallel execution mode for APIs of different stream.
5. Synchronize streams.

   To ensure that the asynchronous execution result is complete when it is used as the input parameter of the next API, call the member function of Synchronize() to explicitly synchronize streams.

6. Destroys streams.

   When the service processes or streams end, destroy the streams by calling the member function of DestroyAscendStream(). Otherwise, streams may be used up.

7. Deinitialize the initialized global resources by calling MxDeInit().

Sample Code

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36

// Initialization
MxInit();
{
        // The user creates the stream of the target deviceId.
        AscendStream stream(deviceId);
        stream.CreateAscendStream();
        // Create an instance of the image decoding class.
        ImageProcessor imageProcessor(deviceId);
        // Instance of the decoded Image class
        Image decodedImage;
        // Perform decoding based on the image path.
        APP_ERROR ret = imageProcessor.Decode(imagePath, decodedImage);
        // Image class instance after resizing
        Image resizedImage;
        // Call the resizing API asynchronously.
        ret = imageProcessor.Resize(decodedImage, Size(416, 416), resizedImage, Interpolation::HUAWEI_HIGH_ORDER_FILTER, stream);
        // Call the synchronous API to explicitly ensure that the resizing is complete when subsequent APIs are called.
        stream.Synchronize();
        // Convert the Image class to the Tensor class.
        Tensor tensorImg = resizedImage.ConvertToTensor();
        // YOLOv3 model inference
        string yoloPath = "./model/yolov3_tf_bs1_fp16.om";
        Model yoloV3(yoloPath, deviceId);
        // Construct the inference input and output (single batch).
        vector<Tensor> yoloV3Inputs = {tensorImg};
        vector<Tensor> yoloV3Outputs = {};
        // Start asynchronous inference.
        ret = yoloV3.Infer(yoloV3Inputs, yoloV3Outputs, stream);
        // Call the synchronous API to explicitly ensure that the resizing is complete when subsequent APIs are called.
        // You can also choose to serialize other asynchronous APIs in the stream before synchronization.
        stream.Synchronize();
        // Call DestroyAscendStream to destroy the streams.
        stream.DestroyAscendStream();
}
// Deinitialization
MxDeInit();