Dynamic AIPP (Single Dynamic AIPP Input)

API Call Sequence

In the dynamic AIPP scenario, the model inference process is similar to the Model Inference with Static-Shape Inputs process, including initialization and deinitialization, runtime resource allocation and release, model building, model loading, model execution, and model unloading.

This section describes the differences between model inference and Model Inference with Static-Shape Inputs in the dynamic AIPP scenario.

• During model building: You need to set the parameters related to dynamic AIPP.

  When building a model, you need to configure the dynamic AIPP mode by setting the insert_op_conf parameter of the ATC tool. For details about ATC parameters, see Command-Line Options > Advanced Options > Feature Options > --insert_op_conf in ATC.

  After model building, the AIPP inputs are added to the generated OM model. During model inference, the AIPP input values are provided.

  Assume that the AIPP configuration of input a is dynamic. In the generated .om model, input b is added to describe the AIPP configuration of input a. For details about how to prepare the data structure of input a for model execution, see Preparing Input/Output Data Structure for Model Execution. For details about how to prepare the data structure of input b and set the data of input b, see the following description.

• Before model inference
  • Prepare the data structure of the dynamic AIPP inputs.
    1. Before allocating memory for the dynamic AIPP input, call aclmdlGetAippDataSize to obtain the memory size.

       In the earlier version, the method of obtaining the memory size by calling aclmdlGetInputSizeByIndex is still supported. However, in this method, the obtained memory size may be 0 when the batch size is not fixed. In this case, you need to estimate the memory size. If the aclmdlGetInputSizeByIndex API is called to obtain the memory size, the aclmdlGetInputIndexByName API needs to be called in advance to obtain the index that identifies the input in the model based on the input name (fixed as the ACL_DYNAMIC_AIPP_NAME macro, indicating ascend_dynamic_aipp_data).

    2. Call aclrtMalloc to apply for memory based on the size specified by 1.

       Do not initialize the buffer manually; otherwise, the service will be unavailable. After the API calls described in 2, the system automatically initializes the buffer.

    3. Create data of the aclDataBuffer type by calling aclCreateDataBuffer. The data is used to store the memory address and memory size of the dynamic AIPP input data.
    4. Call aclmdlCreateDataset to create data of the aclmdlDataset type, and call aclmdlAddDatasetBuffer to add data of the aclDataBuffer type to the data of the aclmdlDataset type.
  • Set dynamic AIPP parameters.
    Figure 1 API call sequence
    

    1. Call aclmdlGetInputIndexByName to obtain the index that identifies the input in the model based on the input name (fixed at ACL_DYNAMIC_AIPP_NAME).
    2. Set dynamic AIPP parameters.
       1. Call aclmdlCreateAIPP to create the aclmdlAIPP type.
       2. Call the APIs of the aclmdlAIPP data type to set dynamic AIPP attributes as required.
       3. In the dynamic AIPP scenario, the aclmdlSetAIPPSrcImageSize API (for setting the width and height of the source image) must be called.
       4. Call aclmdlSetInputAIPP to set the dynamic AIPP data for model inference.
       5. Call the aclmdlDestroyAIPP API to destroy the aclmdlAIPP type in a timely manner.

Sample Code

The following is a code example of key steps of model inference. It is for reference only and cannot be directly copied for compilation and running. After APIs are called, you need to add exception handling branches and record error logs and info logs.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44

......

// 1. Load the model and set dynamic AIPP parameters after the model is successfully loaded.
// ......

// 2. Prepare the model description information modelDesc_, model input data input_, and model output data output_.

// 3. Customize a function to set dynamic AIPP parameters.
int  ModelSetDynamicAIPP()
{
    //3.1 Obtain the index of the dynamic AIPP input.
    size_t index;
    //modelDesc_ is of type aclmdlCreateDesc for the model description, which is obtained based on the ID of the model that is successfully loaded in step 1.
    aclError ret = aclmdlGetInputIndexByName(modelDesc_, ACL_DYNAMIC_AIPP_NAME, &index);

    //3.2 Set dynamic AIPP attributes.
    uint64_t batchNumber = 1;
    aclmdlAIPP *aippDynamicSet = aclmdlCreateAIPP(batchNumber);
    ret = aclmdlSetAIPPSrcImageSize(aippDynamicSet, 256, 224);
    ret = aclmdlSetAIPPInputFormat(aippDynamicSet, ACL_YUV420SP_U8);
    ret = aclmdlSetAIPPCscParams(aippDynamicSet, 1, 256, 443, 0, 256, -86, -178, 256, 0, 350, 0, 0, 0, 0, 128, 128);
    ret = aclmdlSetAIPPRbuvSwapSwitch(aippDynamicSet, 0);
    ret = aclmdlSetAIPPDtcPixelMean(aippDynamicSet, 0, 0, 0, 0, 0);
    ret = aclmdlSetAIPPDtcPixelMin(aippDynamicSet, 0, 0, 0, 0, 0);
    ret = aclmdlSetAIPPPixelVarReci(aippDynamicSet, 1.0, 1.0, 1.0, 1.0, 0);
    ret = aclmdlSetAIPPCropParams(aippDynamicSet, 1, 2, 2, 224, 224, 0);
    ret = aclmdlSetInputAIPP(modelId_, input_, index, aippDynamicSet);
    ret = aclmdlDestroyAIPP(aippDynamicSet);
		
    // ......
}
// 4. Customize a function to execute the model.
int ModelExecute(int index)
{
        aclError ret;
        //6.1 Call the user-defined function to set the dynamic AIPP attributes.
	ret = ModelSetDynamicAIPP();
        //6.2 Execute the model. modelId_ indicates the ID of a successfully loaded model, input_ indicates the model inputs, and output_ indicates the model outputs.
        ret = aclmdlExecute(modelId_, input_, output_);
        // ......
}

// 5. Process the model inference result.
......