Documentation
Download

Typical Functions of VPC

This section takes image cropping and resizing as an example to explain the API call sequence when the VPC processes images and provides sample code of typical functions to help you better understand the API call sequence.

Vision Preprocessing Core (VPC) supports image cropping, resizing, pasting, and format conversion. For details about the VPC function and constraints, see DVPP Media Acceleration Library.

API Call Sequence (for Cropping and Resizing)

Figure 1 Cropping and resizing

The key APIs are described as follows (taking cropping and resizing as an example):

  1. Call aclInit to initialize the system.
  2. Call aclrtSetDevice to specify the compute device.
  3. aclrtCreateStream: creates a stream.
  4. Call acldvppCreateChannel to create a channel for image data processing.

    Before creating a channel, call <cf id="Bold">acldvppCreateChannelDesc</cf> to create the channel description.

  5. Call acldvppCreateRoiConfig and acldvppCreateResizeConfig to create the crop ROI position and resizing settings, respectively, and call acldvppSetResizeConfigInterpolation to specify the resizing method.
  6. Before cropping and resizing, call acldvppMalloc to allocate memory on the device for storing the input or output data.
  7. Perform cropping and resizing.
    • About cropping:
      • Call the asynchronous API acldvppVpcCropAsync to crop a selected ROI from the input image and load the cropped image to the output buffer.

        The cropped image will be resized again if cropArea is different from that of the output image.

      • The system also provides the acldvppVpcCropAndPasteAsync asynchronous API to crop the input image based on the specified area and paste the cropped image to the specified position of the target image as the output image.
        • The cropped image will be resized again if the size of the cropArea is inconsistent with that of the pasteArea.
        • To paste the pasteArea on a canvas loaded to the output buffer, modify the code logic as follows: Allocate the output buffer and then load a canvas to the allocated buffer.
    • About resizing:
      • Call the asynchronous API <cf id="Bold">acldvppVpcResizeAsync</cf> to resize the input image to the size of the output image.
      • Calculate the buffer for storing the resized image based on the YUV420SP format as follows: widthStride x heightStride x 3/2
    • For asynchronous APIs, call aclrtSynchronizeStream to block program running until all tasks in the specified stream are complete.
  8. Call acldvppFree to free the input and output buffers.
  9. Call acldvppDestroyRoiConfig and acldvppDestroyResizeConfig to destroy the crop ROI position and resizing settings, respectively.
  10. Call acldvppDestroyChannel to destroy the image data processing channel.

    After destroying the channel, call <cf id="Bold">acldvppDestroyChannelDesc</cf> to destroy the channel description.

  11. aclrtDestroyStream: destroys a stream.
  12. Call aclrtResetDevice to reset the device and free the resources on the device.
  13. Call aclFinalize to deinitialize the system and free the resources used by the acl API in the process.

Sample Code for Image Resizing

The following is a code example of key steps of the image resizing function. It is for reference only and cannot be directly copied for compilation and running. After APIs are called, you need to add exception handling branches and record error logs and info logs.

You can click vpc_resnet50_imagenet_classification to obtain the sample.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
// 1. Create the image resizing configuration data and specify the resizing algorithm.
// resizeConfig_ is of type acldvppResizeConfig.
acldvppResizeConfig *resizeConfig_ = acldvppCreateResizeConfig();
aclError ret = acldvppSetResizeConfigInterpolation(resizeConfig_, 0);

// 2. Create the description of the image data processing channel. dvppChannelDesc_ is of the acldvppChannelDesc type.
dvppChannelDesc_ = acldvppCreateChannelDesc();

// 3. Create a channel for processing image data.
ret = acldvppCreateChannel(dvppChannelDesc_);

// 4. Allocate input buffer (differentiated by running status).
// Call aclrtGetRunMode to obtain the run mode of the software stack. If ACL_DEVICE is returned, allocate and use the device buffer. If ACL_HOST is returned, transfer the input image data to the device by using the aclrtMemcpy call. After the data transfer is complete, the buffer needs to be freed in a timely manner.
aclrtRunMode runMode;
ret = aclrtGetRunMode(&runMode);
// inputPicWidth and inputPicHeight respectively indicate the width and height of an image after alignment. The following uses a YUV420SP image as an example.
uint32_t resizeInBufferSize = inputPicWidth * inputPicHeight * 3 / 2;
if(runMode == ACL_HOST){ 
    //Allocate host buffer vpcInHostBuffer.
    void* vpcInHostBuffer = nullptr;
    vpcInHostBuffer = malloc(resizeInBufferSize);
    // Load the input image into the buffer. The ReadPicFile function is defined by the user.
    ReadPicFile(picName, vpcInHostBuffer, resizeInBufferSize);
    // Allocate the device buffer resizeInDevBuffer_.
    ret = acldvppMalloc(&resizeInDevBuffer_, resizeInBufferSize);
    // Transfer the input image data to the device by using the aclrtMemcpy call.
    ret = aclrtMemcpy(resizeInDevBuffer_, resizeInBufferSize, vpcInHostBuffer, resizeInBufferSize, ACL_MEMCPY_HOST_TO_DEVICE);
    // Free the buffer in a timely manner after data transfer is complete.
    free(vpcInHostBuffer);
} else {
    // Allocate input buffer resizeInDevBuffer_ on the device.
    ret = acldvppMalloc(&resizeInDevBuffer_, resizeInBufferSize);
    // Load the input image into the buffer. The ReadPicFile function is defined by the user.
    ReadPicFile(picName, resizeInDevBuffer_, resizeInBufferSize);
}

// 5. Allocate the resizing output buffer resizeOutBufferDev_. The memory size resizeOutBufferSize_ is calculated based on the formula.
// outputPicWidth and outputPicHeight respectively indicate the width and height after image alignment. The following uses a YUV420SP image as an example.
uint32_t resizeOutBufferSize_ = outputPicWidth * outputPicHeight * 3 / 2;
ret = acldvppMalloc(&resizeOutBufferDev_, resizeOutBufferSize_);

// 6. Create the description of the input image and set the attribute values.
// resizeInputDesc_ is of type acldvppPicDesc.
resizeInputDesc_ = acldvppCreatePicDesc();
acldvppSetPicDescData(resizeInputDesc_, resizeInDevBuffer_);
acldvppSetPicDescFormat(resizeInputDesc_, PIXEL_FORMAT_YUV_SEMIPLANAR_420);
acldvppSetPicDescWidth(resizeInputDesc_, inputWidth_);
acldvppSetPicDescHeight(resizeInputDesc_, inputHeight_);
acldvppSetPicDescWidthStride(resizeInputDesc_, inputWidthStride);
acldvppSetPicDescHeightStride(resizeInputDesc_, inputHeightStride);
acldvppSetPicDescSize(resizeInputDesc_, resizeInBufferSize);

// 7. Create the description of the output image and set the attributes.
// If the resized output image is used as the input for model inference, the output width and height must meet the requirements of the model.
// resizeOutputDesc_ is of type acldvppPicDesc.
resizeOutputDesc_ = acldvppCreatePicDesc();
acldvppSetPicDescData(resizeOutputDesc_, resizeOutBufferDev_);
acldvppSetPicDescFormat(resizeOutputDesc_, PIXEL_FORMAT_YUV_SEMIPLANAR_420); 
acldvppSetPicDescWidth(resizeOutputDesc_, resizeOutputWidth_);
acldvppSetPicDescHeight(resizeOutputDesc_, resizeOutputHeight_);
acldvppSetPicDescWidthStride(resizeOutputDesc_, resizeOutputWidthStride);
acldvppSetPicDescHeightStride(resizeOutputDesc_, resizeOutputHeightStride);
acldvppSetPicDescSize(resizeOutputDesc_, resizeOutBufferSize_);

// 8. Perform asynchronous resizing and call aclrtSynchronizeStream to block program running until all tasks in the specified stream are complete.
ret = acldvppVpcResizeAsync(dvppChannelDesc_, resizeInputDesc_,
        resizeOutputDesc_, resizeConfig_, stream_);
ret = aclrtSynchronizeStream(stream_);

// 9. After the resizing is complete, release the resources, including the description of the input and output images and the input and output buffers.
acldvppDestroyPicDesc(resizeInputDesc_);
acldvppDestroyPicDesc(resizeOutputDesc_);

if(runMode == ACL_HOST){ 
    // In this mode, the processing result is on the device. Therefore, you need to call the memory copy API to transfer the result data, and then free the device buffer.
    // Allocate host buffer vpcOutHostBuffer.
    void* vpcOutHostBuffer = nullptr;
    vpcOutHostBuffer = malloc(resizeOutBufferSize_);
    // Transfer the processing result of the device to the host by using the aclrtMemcpy call.
    ret = aclrtMemcpy(vpcOutHostBuffer, resizeOutBufferSize_, resizeOutBufferDev_, resizeOutBufferSize_, ACL_MEMCPY_DEVICE_TO_HOST);
    // Free the input and output buffer on the device.
    (void)acldvppFree(resizeInDevBuffer_);
    (void)acldvppFree(resizeOutBufferDev_);
    // Free the buffer after the data is used.
    free(vpcOutHostBuffer);
} else { 
    // The process is running on the device, and the processing result is also on the device. Free the device buffer when the data is no longer needed.
    (void)acldvppFree(resizeInDevBuffer_);
    (void)acldvppFree(resizeOutBufferDev_);
}
acldvppDestroyChannel(dvppChannelDesc_);
(void)acldvppDestroyChannelDesc(dvppChannelDesc_);
dvppChannelDesc_ = nullptr;

 ....

Sample Code for Format Conversion

You can convert formats by using either of the following methods:
  • When the corresponding API (such as acldvppVpcCropAsync) is called to implement functions such as image cropping and resizing, you can set different formats for the input and output images to convert the image format.
  • If only image format conversion is required, you can directly call acldvppVpcConvertColorAsync.

The following is a code example of key steps of the format conversion function. It is for reference only and cannot be directly copied for compilation and running. After APIs are called, you need to add exception handling branches and record error logs and info logs.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
// 1. Create the description of the image data processing channel. dvppChannelDesc_ is of the acldvppChannelDesc type.
dvppChannelDesc_ = acldvppCreateChannelDesc();

// 2. Create a channel for processing image data.
aclError ret = acldvppCreateChannel(dvppChannelDesc_);

// 3. Allocate input buffer (differentiated by running status).
// Call aclrtGetRunMode to obtain the run mode of the software stack. If ACL_DEVICE is returned, allocate and use the device buffer. If ACL_HOST is returned, transfer the input image data to the device by using the aclrtMemcpy call. After the data transfer is complete, the buffer needs to be freed in a timely manner.
aclrtRunMode runMode;
ret = aclrtGetRunMode(&runMode);
// inputPicWidth and inputPicHeight respectively indicate the width and height of an image after alignment. The following uses a YUV420SP image as an example.
uint32_t inBufferSize = inputPicWidth * inputPicHeight * 3 / 2;
if(runMode == ACL_HOST){ 
    // Allocate host buffer vpcInHostBuffer.
    void* vpcInHostBuffer = nullptr;
    vpcInHostBuffer = malloc(inBufferSize);
    // Load the input image into the buffer. The ReadPicFile function is defined by the user.
    ReadPicFile(picName, vpcInHostBuffer, inBufferSize);
    // Allocate the device buffer inDevBuffer_.
    ret = acldvppMalloc(&inDevBuffer_, inBufferSize);
    // Transfer the input image data to the device by using the aclrtMemcpy call.
    ret = aclrtMemcpy(inDevBuffer_, inBufferSize, vpcInHostBuffer, inBufferSize, ACL_MEMCPY_HOST_TO_DEVICE);
    // Free the buffer in a timely manner after data transfer is complete.
    free(vpcInHostBuffer);
} else {
    // Allocate input buffer inDevBuffer_ on the device.
    ret = acldvppMalloc(&inDevBuffer_, inBufferSize);
    // Load the input image into the buffer. The ReadPicFile function is defined by the user.
    ReadPicFile(picName, inDevBuffer_, inBufferSize);
}

// 4. Allocate the CSC output buffer outBufferDev_. The buffer size outBufferSize_ is calculated based on the formula.
// outputPicWidth and outputPicHeight respectively indicate the width and height after image alignment. The following uses a YUV420SP image as an example.
uint32_t outBufferSize_ = outputPicWidth * outputPicHeight * 3 / 2;
ret = acldvppMalloc(&outBufferDev_, outBufferSize_);

// 5. Create the description of the input image for CSC and set the attribute values.
// inputDesc_ is of type acldvppPicDesc.
inputDesc_ = acldvppCreatePicDesc();
acldvppSetPicDescData(inputDesc_, inDevBuffer_);
acldvppSetPicDescFormat(inputDesc_, PIXEL_FORMAT_YUV_SEMIPLANAR_420);
acldvppSetPicDescWidth(inputDesc_, inputWidth_);
acldvppSetPicDescHeight(inputDesc_, inputHeight_);
acldvppSetPicDescWidthStride(inputDesc_, inputWidthStride);
acldvppSetPicDescHeightStride(inputDesc_, inputHeightStride);
acldvppSetPicDescSize(inputDesc_, inBufferSize);

// 6. Create the description of the output image for CSC and set the attribute values. The output width and height must be the same as those of the input.
// If the CSC output image is used as the input for model inference, the output width and height must meet the requirements of the model.
// outputDesc_ is of type acldvppPicDesc. 
outputDesc_= acldvppCreatePicDesc();
acldvppSetPicDescData(outputDesc_, outBufferDev_);
acldvppSetPicDescFormat(outputDesc_, PIXEL_FORMAT_YUV_400); 
acldvppSetPicDescWidth(outputDesc_, outputWidth_);
acldvppSetPicDescHeight(outputDesc_, outputHeight_);
acldvppSetPicDescWidthStride(outputDesc_, outputWidthStride);
acldvppSetPicDescHeightStride(outputDesc_, outputHeightStride);
acldvppSetPicDescSize(outputDesc_, outBufferSize_);

// 7. Perform asynchronous CSC and call aclrtSynchronizeStream to block program running until all tasks in the specified stream are complete.
ret = acldvppVpcConvertColorAsync(dvppChannelDesc_, inputDesc_, outputDesc_, stream_);
ret = aclrtSynchronizeStream(stream_);

// 8. After CSC is complete, release resources, including the description of the input and output images and the input and output buffers.
acldvppDestroyPicDesc(inputDesc_);
acldvppDestroyPicDesc(outputDesc_);

if(runMode == ACL_HOST){ 
    // In this mode, the processing result is on the device. Therefore, you need to call the memory copy API to transfer the result data, and then free the device buffer.
    // Allocate host buffer vpcOutHostBuffer.
    void* vpcOutHostBuffer = nullptr;
    vpcOutHostBuffer = malloc(outBufferSize_);
    // Transfer the processing result of the device to the host by using the aclrtMemcpy call.
    ret = aclrtMemcpy(vpcOutHostBuffer, outBufferSize_, outBufferDev_, outBufferSize_, ACL_MEMCPY_DEVICE_TO_HOST);
    // Free the input and output buffer on the device.
    (void)acldvppFree(inDevBuffer_);
    (void)acldvppFree(outBufferDev_);
    // Free the buffer after the data is used.
    free(vpcOutHostBuffer);
} else { 
    // The process is running on the device, and the processing result is also on the device. Free the device buffer when the data is no longer needed.
    (void)acldvppFree(inDevBuffer_);
    (void)acldvppFree(outBufferDev_);
}
acldvppDestroyChannel(dvppChannelDesc_);
(void)acldvppDestroyChannelDesc(dvppChannelDesc_);
dvppChannelDesc_ = nullptr;

....

Sample Code for Image Cropping and Resizing (Single-Image, Single-ROI)

Call the asynchronous API acldvppVpcCropResizeAsync to crop the input image based on the specified ROI and save the cropped image to the output buffer as the output image. In addition, the resizing algorithm can be specified. The cropped image will be resized again if cropArea is different from that of the output image.

The following is a code example of key steps for cropping and resizing. It is for reference only and cannot be directly copied for compilation and running. After APIs are called, you need to add exception handling branches and record error logs and info logs.

You can click vpc_jpeg_resnet50_imagenet_classification to obtain the sample.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
// 1. Create the resizing config data and specify the position of the crop ROI.
// resizeConfig_ is of type acldvppResizeConfig.
resizeConfig_ = acldvppCreateResizeConfig();
aclError ret = acldvppSetResizeConfigInterpolation(resizeConfig_, 0);
// cropArea_ is of type acldvppRoiConfig.
cropArea_ = acldvppCreateRoiConfig(550, 749, 480, 679);

// 2. Create the description of the image data processing channel. dvppChannelDesc_ is of the acldvppChannelDesc type.
dvppChannelDesc_ = acldvppCreateChannelDesc();

// 3. Create a channel for processing image data.
ret = acldvppCreateChannel(dvppChannelDesc_);

// 4. Allocate input buffer (differentiated by running status).
// Call aclrtGetRunMode to obtain the run mode of the software stack. If ACL_DEVICE is returned, allocate and use the device buffer. If ACL_HOST is returned, transfer the input image data to the device by using the aclrtMemcpy call. After the data transfer is complete, the buffer needs to be freed in a timely manner.
aclrtRunMode runMode;
ret = aclrtGetRunMode(&runMode);
// inputPicWidth and inputPicHeight respectively indicate the width and height of an image after alignment. The following uses a YUV420SP image as an example.
uint32_t cropInBufferSize = inputPicWidth * inputPicHeight * 3 / 2;
if(runMode == ACL_HOST){ 
    // Allocate host buffer cropInHostBuffer.
    void* cropInHostBuffer = nullptr;
    cropInHostBuffer = malloc(cropInBufferSize);
    // Load the input image into the buffer. The ReadPicFile function is defined by the user.
    ReadPicFile(picName, cropInHostBuffer, cropInBufferSize);
    // Allocate the device buffer cropInDevBuffer_.
    ret = acldvppMalloc(&cropInDevBuffer_, cropInBufferSize);
    // Transfer the image data to the device by using the aclrtMemcpy call.
    ret = aclrtMemcpy(cropInDevBuffer_, cropInBufferSize, cropInHostBuffer, cropInBufferSize, ACL_MEMCPY_HOST_TO_DEVICE);
    // Free the buffer in a timely manner after data transfer is complete.
    free(cropInHostBuffer);
} else { 
    // Allocate input buffer cropInDevBuffer_ on the device.
    ret = acldvppMalloc(&cropInDevBuffer_, cropInBufferSize);
    // Load the input image into the buffer. The ReadPicFile function is defined by the user.
    ReadPicFile(picName, cropInDevBuffer_, cropInBufferSize);
}

// 5. Allocate the device output buffer cropOutBufferDev_. The buffer size cropOutBufferSize_ is calculated based on the formula.
// outputPicWidth and outputPicHeight respectively indicate the width and height after image alignment. The following uses a YUV420SP image as an example.
uint32_t cropOutBufferSize_ = outputPicWidth * outputPicHeight * 3 / 2;
ret = acldvppMalloc(&cropOutBufferDev_, cropOutBufferSize_);

// 6. Create the description of the input image and set the attribute values. cropInputDesc_ is of the acldvppPicDesc type.
cropInputDesc_ = acldvppCreatePicDesc();
acldvppSetPicDescData(cropInputDesc_, cropInDevBuffer_);
acldvppSetPicDescFormat(cropInputDesc_, PIXEL_FORMAT_YUV_SEMIPLANAR_420);
acldvppSetPicDescWidth(cropInputDesc_, inputWidth_);
acldvppSetPicDescHeight(cropInputDesc_, inputHeight_);
acldvppSetPicDescWidthStride(cropInputDesc_, inputWidthStride);
acldvppSetPicDescHeightStride(cropInputDesc_, inputHeightStride);
acldvppSetPicDescSize(cropInputDesc_, cropInBufferSize);

// 7. Create the description of the output image and set the attribute values. cropOutputDesc_ is of the acldvppPicDesc type.
// If the cropped output image is used as the input for model inference, the output width and height must meet the requirements of the model.
cropOutputDesc_ = acldvppCreatePicDesc();
acldvppSetPicDescData(cropOutputDesc_, cropOutBufferDev_);
acldvppSetPicDescFormat(cropOutputDesc_, PIXEL_FORMAT_YUV_SEMIPLANAR_420); 
acldvppSetPicDescWidth(cropOutputDesc_, OutputWidth_);
acldvppSetPicDescHeight(cropOutputDesc_, OutputHeight_);
acldvppSetPicDescWidthStride(cropOutputDesc_, OutputWidthStride);
acldvppSetPicDescHeightStride(cropOutputDesc_, OutputHeightStride);
acldvppSetPicDescSize(cropOutputDesc_, cropOutBufferSize_);

// 8. Perform asynchronous cropping and resizing and call aclrtSynchronizeStream to block program running until all tasks in the specified stream are complete.
ret = acldvppVpcCropResizeAsync(dvppChannelDesc_, cropInputDesc_,
        cropOutputDesc_, cropArea_, resizeConfig_, stream_);
ret = aclrtSynchronizeStream(stream_);

// 9. After the cropping and pasting are complete, release the resources, including the description of the input and output images, input and output buffers, channel description, and channel.
acldvppDestroyRoiConfig(cropArea_);
acldvppDestroyResizeConfig(resizeConfig_);
acldvppDestroyPicDesc(cropInputDesc_);
acldvppDestroyPicDesc(cropOutputDesc_);
if(runMode == ACL_HOST){ 
    // In this mode, the processing result is on the device. Therefore, you need to call the memory copy API to transfer the result data, and then free the device buffer.
    // Allocate host buffer cropOutHostBuffer.
    void* cropOutHostBuffer = nullptr;
    cropOutHostBuffer = malloc(cropOutBufferSize_);
    // Transfer the processing result of the device to the host by using the aclrtMemcpy call.
    ret = aclrtMemcpy(cropOutHostBuffer, cropOutBufferSize_, cropOutBufferDev_, cropOutBufferSize_, ACL_MEMCPY_DEVICE_TO_HOST);
    // Free the input and output buffer on the device.
    (void)acldvppFree(cropInDevBuffer_);
    (void)acldvppFree(cropOutBufferDev_);
    // Free the buffer after the data is used.
    free(cropOutHostBuffer);
} else { 
    // The process is running on the device, and the processing result is also on the device. Free the device buffer when the data is no longer needed.
    (void)acldvppFree(cropInDevBuffer_);
    (void)acldvppFree(cropOutBufferDev_);
}
acldvppDestroyChannel(dvppChannelDesc_);
(void)acldvppDestroyChannelDesc(dvppChannelDesc_);
dvppChannelDesc_ = nullptr;

....

Sample Code for Image Cropping, Resizing, and Pasting (Single-Image, Single-ROI)

Call the asynchronous API acldvppVpcCropResizePasteAsync to crop the input image based on the specified area and paste the cropped image to the specified position of the target image as the output image. In addition, you can specify the resizing algorithm. The cropped image will be resized again if the size of the cropArea is inconsistent with that of the pasteArea. To paste the pasteArea on a canvas loaded to the output buffer, modify the code logic as follows: Allocate the output buffer and then load a canvas to the allocated buffer.

The following is a code example of the key steps of cropping, pasting, and resizing. It is for reference only and cannot be directly copied for compilation and running. After APIs are called, you need to add exception handling branches and record error logs and info logs.

You can click vpc_jpeg_resnet50_imagenet_classification to obtain the sample.

  1
  2
  3
  4
  5
  6
  7
  8
  9
 10
 11
 12
 13
 14
 15
 16
 17
 18
 19
 20
 21
 22
 23
 24
 25
 26
 27
 28
 29
 30
 31
 32
 33
 34
 35
 36
 37
 38
 39
 40
 41
 42
 43
 44
 45
 46
 47
 48
 49
 50
 51
 52
 53
 54
 55
 56
 57
 58
 59
 60
 61
 62
 63
 64
 65
 66
 67
 68
 69
 70
 71
 72
 73
 74
 75
 76
 77
 78
 79
 80
 81
 82
 83
 84
 85
 86
 87
 88
 89
 90
 91
 92
 93
 94
 95
 96
 97
 98
 99
100
101
102
// 1. Create the resizing config data and specify the position of the crop ROI and the position of the paste ROI.
// resizeConfig_ is of type acldvppResizeConfig.
resizeConfig_ = acldvppCreateResizeConfig();
aclError ret = acldvppSetResizeConfigInterpolation(resizeConfig_, 0);
// cropArea_ and pasteArea_ are of type acldvppRoiConfig.
cropArea_ = acldvppCreateRoiConfig(512, 711, 512, 711);
pasteArea_ = acldvppCreateRoiConfig(16, 215, 16, 215);

// 2. Create the description of the image data processing channel. dvppChannelDesc_ is of the acldvppChannelDesc type.
dvppChannelDesc_ = acldvppCreateChannelDesc();

// 3. Create a channel for processing image data.
ret = acldvppCreateChannel(dvppChannelDesc_);

// 4. Allocate input buffer (differentiated by running status).
// Call aclrtGetRunMode to obtain the run mode of the software stack. If ACL_DEVICE is returned, allocate and use the device buffer. If ACL_HOST is returned, transfer the input image data to the device by using the aclrtMemcpy call. After the data transfer is complete, the buffer needs to be freed in a timely manner.
aclrtRunMode runMode;
ret = aclrtGetRunMode(&runMode);
// inputPicWidth and inputPicHeight respectively indicate the width and height of an image after alignment. The following uses a YUV420SP image as an example.
uint32_t vpcInBufferSize = inputPicWidth * inputPicHeight * 3 / 2;
if(runMode == ACL_HOST){ 
    //Allocate host buffer vpcInHostBuffer.
    void* vpcInHostBuffer = nullptr;
    vpcInHostBuffer = malloc(vpcInBufferSize);
    // Load the input image into the buffer. The ReadPicFile function is defined by the user.
    ReadPicFile(picName, vpcInHostBuffer, vpcInBufferSize);
    // Allocate the device buffer vpcInDevBuffer_.
    ret = acldvppMalloc(&vpcInDevBuffer_, vpcInBufferSize);
    // Transfer the input image data to the device by using the aclrtMemcpy call.
    ret = aclrtMemcpy(vpcInDevBuffer_, vpcInBufferSize, vpcInHostBuffer, vpcInBufferSize, ACL_MEMCPY_HOST_TO_DEVICE);
    // Free the buffer in a timely manner after data transfer is complete.
    free(vpcInHostBuffer);
} else {
    // Allocate input buffer vpcInDevBuffer_ on the device.
    ret = acldvppMalloc(&vpcInDevBuffer_, vpcInBufferSize);
    // Load the input image into the buffer. The ReadPicFile function is defined by the user.
    ReadPicFile(picName, vpcInDevBuffer_, vpcInBufferSize);
}

// 5. Allocate the output buffer vpcOutBufferDev_. The memory size vpcOutBufferSize_ is calculated based on the formula.
// outputPicWidth and outputPicHeight respectively indicate the width and height after image alignment. The following uses a YUV420SP image as an example.
uint32_t vpcOutBufferSize_ = outputPicWidth * outputPicHeight * 3 / 2;
ret = acldvppMalloc(&vpcOutBufferDev_, vpcOutBufferSize_);

// 6. Create the description of the input image and set the attribute values.
// In this sample, the decoding output buffer is used as the input of image cropping and pasting. vpcInputDesc_ is of type acldvppPicDesc.
vpcInputDesc_ = acldvppCreatePicDesc();
acldvppSetPicDescData(vpcInputDesc_, decodeOutBufferDev_); 
acldvppSetPicDescFormat(vpcInputDesc_, PIXEL_FORMAT_YUV_SEMIPLANAR_420);
acldvppSetPicDescWidth(vpcInputDesc_, inputWidth_);
acldvppSetPicDescHeight(vpcInputDesc_, inputHeight_);
acldvppSetPicDescWidthStride(vpcInputDesc_, jpegOutWidthStride);
acldvppSetPicDescHeightStride(vpcInputDesc_, jpegOutHeightStride);
acldvppSetPicDescSize(vpcInputDesc_, jpegOutBufferSize);

// 7. Create the description of the output image and set the attribute values.
// If the cropped and pasted output image is used as the input for model inference, the output width and height must meet the requirements of the model.
// vpcOutputDesc_ is of type acldvppPicDesc.
vpcOutputDesc_ = acldvppCreatePicDesc();
acldvppSetPicDescData(vpcOutputDesc_, vpcOutBufferDev_);
acldvppSetPicDescFormat(vpcOutputDesc_, PIXEL_FORMAT_YUV_SEMIPLANAR_420);
acldvppSetPicDescWidth(vpcOutputDesc_, dvppOutWidth);
acldvppSetPicDescHeight(vpcOutputDesc_, dvppOutHeight);
acldvppSetPicDescWidthStride(vpcOutputDesc_, dvppOutWidthStride);
acldvppSetPicDescHeightStride(vpcOutputDesc_, dvppOutHeightStride);
acldvppSetPicDescSize(vpcOutputDesc_, vpcOutBufferSize_);

// 8. Perform asynchronous cropping, pasting, and resizing, and call aclrtSynchronizeStream to block program running until all tasks in the specified stream are complete.
ret = acldvppVpcCropResizePasteAsync(dvppChannelDesc_, vpcInputDesc_,
        vpcOutputDesc_, cropArea_, pasteArea_, resizeConfig_, stream_);
ret = aclrtSynchronizeStream(stream_);

// 9. After the cropping and pasting are complete, release the resources, including the description of the input and output images, input and output buffers, channel description, and channel.
acldvppDestroyRoiConfig(cropArea_);
acldvppDestroyRoiConfig(pasteArea_);
acldvppDestroyResizeConfig(resizeConfig_);
acldvppDestroyPicDesc(vpcInputDesc_);
acldvppDestroyPicDesc(vpcOutputDesc_);

if(runMode == ACL_HOST){ 
    // In this mode, the processing result is on the device. Therefore, you need to call the memory copy API to transfer the result data, and then free the device buffer.
    // Allocate host buffer vpcOutHostBuffer.
    void* vpcOutHostBuffer = nullptr;
    vpcOutHostBuffer = malloc(vpcOutBufferSize_);
    // Transfer the processing result of the device to the host by using the aclrtMemcpy call.
    ret = aclrtMemcpy(vpcOutHostBuffer, vpcOutBufferSize_, vpcOutBufferDev_, vpcOutBufferSize_, ACL_MEMCPY_DEVICE_TO_HOST);
    // Free the input and output buffer on the device.
    (void)acldvppFree(vpcInDevBuffer_);
    (void)acldvppFree(vpcOutBufferDev_);
    // Free the buffer after the data is used.
    free(vpcOutHostBuffer);
} else { 
    // The process is running on the device, and the processing result is also on the device. Free the device buffer when the data is no longer needed.
    (void)acldvppFree(vpcInDevBuffer_);
    (void)acldvppFree(vpcOutBufferDev_);
}

acldvppDestroyChannel(dvppChannelDesc_);
(void)acldvppDestroyChannelDesc(dvppChannelDesc_);
dvppChannelDesc_ = nullptr;

....

Sample Code for Image Cropping and Pasting (Single-Image, Multi-ROI)

Call the asynchronous API acldvppVpcBatchCropAndPasteAsync to crop the input image based on the specified area and paste the cropped image to the specified position of the target image as the output image. The cropped image will be resized again if the size of the cropArea is inconsistent with that of the pasteArea. To paste the pasteArea on a canvas loaded to the output buffer, modify the code logic as follows: Allocate the output buffer and then load a canvas to the allocated buffer.

The following is a code example of key steps of the cropping and pasting function. It is for reference only and cannot be directly copied for compilation and running. After APIs are called, you need to add exception handling branches and record error logs and info logs.

  1
  2
  3
  4
  5
  6
  7
  8
  9
 10
 11
 12
 13
 14
 15
 16
 17
 18
 19
 20
 21
 22
 23
 24
 25
 26
 27
 28
 29
 30
 31
 32
 33
 34
 35
 36
 37
 38
 39
 40
 41
 42
 43
 44
 45
 46
 47
 48
 49
 50
 51
 52
 53
 54
 55
 56
 57
 58
 59
 60
 61
 62
 63
 64
 65
 66
 67
 68
 69
 70
 71
 72
 73
 74
 75
 76
 77
 78
 79
 80
 81
 82
 83
 84
 85
 86
 87
 88
 89
 90
 91
 92
 93
 94
 95
 96
 97
 98
 99
100
101
102
103
104
105
106
107
108
109
110
111
112
// 1. Specify the position of the batch crop ROI and the position of the batch paste ROI. cropAreas_ and pasteAreas_ are of type acldvppRoiConfig.
acldvppRoiConfig *cropAreas_[2], pasteAreas_[2];
cropAreas_[0] = acldvppCreateRoiConfig(512, 711, 512, 711);
cropAreas_[1] = acldvppCreateRoiConfig(512, 711, 512, 711);
pasteAreas_[0] = acldvppCreateRoiConfig(16, 215, 16, 215);
pasteAreas_[1] = acldvppCreateRoiConfig(16, 215, 16, 215);

// 2. Create the description of the image data processing channel. dvppChannelDesc_ is of the acldvppChannelDesc type.
dvppChannelDesc_ = acldvppCreateChannelDesc();

// 3. Create a channel for processing image data.
aclError ret = acldvppCreateChannel(dvppChannelDesc_);

// 4. Allocate input buffer (differentiated by running status).
// Call aclrtGetRunMode to obtain the run mode of the software stack. If ACL_DEVICE is returned, allocate and use the device buffer. If ACL_HOST is returned, transfer the input image data to the device by using the aclrtMemcpy call. After the data transfer is complete, the buffer needs to be freed in a timely manner.
aclrtRunMode runMode;
ret = aclrtGetRunMode(&runMode);
// inputPicWidth and inputPicHeight respectively indicate the width and height of an image after alignment. The following uses a YUV420SP image as an example.
uint32_t vpcInBufferSize = inputPicWidth * inputPicHeight * 3 / 2;
if(runMode == ACL_HOST){ 
    //Allocate host buffer vpcInHostBuffer.
    void* vpcInHostBuffer = nullptr;
    vpcInHostBuffer = malloc(vpcInBufferSize);
    // Load the input image into the buffer. The ReadPicFile function is defined by the user.
    ReadPicFile(picName, vpcInHostBuffer, vpcInBufferSize);
    // Allocate the device buffer vpcInDevBuffer_.
    ret = acldvppMalloc(&vpcInDevBuffer_, vpcInBufferSize);
    //Transfer the image data from the host to the device by using the aclrtMemcpy call.
    ret = aclrtMemcpy(vpcInDevBuffer_, vpcInBufferSize, vpcInHostBuffer, vpcInBufferSize, ACL_MEMCPY_HOST_TO_DEVICE);
    // Free the buffer in a timely manner after data transfer is complete.
    free(vpcInHostBuffer);
} else {
    // Allocate input buffer vpcInDevBuffer_ on the device.
    ret = acldvppMalloc(&vpcInDevBuffer_, vpcInBufferSize);
    // Load the input image into the buffer. The ReadPicFile function is defined by the user.
    ReadPicFile(picName, vpcInDevBuffer_, vpcInBufferSize);
}

// 5. Allocate the output buffer vpcOutBufferDev_. The memory size vpcOutBufferSize_ is calculated based on the formula.
// outputPicWidth and outputPicHeight respectively indicate the width and height after image alignment. The following uses a YUV420SP image as an example.
uint32_t vpcOutBufferSize_ = outputPicWidth * outputPicHeight * 3 / 2;
ret = acldvppMalloc(&vpcOutBufferDev_, vpcOutBufferSize_);

// 6. Create the description of the input image and set the attribute values.
// In this sample, the decoding output buffer is used as the input of image cropping and pasting. vpcInputDesc_ is of type acldvppPicDesc.
vpcInputBatchDesc_ = acldvppCreateBatchPicDesc(1);
vpcInputDesc_ = acldvppGetPicDesc(vpcInputBatchDesc_, 0);
acldvppSetPicDescData(vpcInputDesc_, decodeOutBufferDev_); 
acldvppSetPicDescFormat(vpcInputDesc_, PIXEL_FORMAT_YUV_SEMIPLANAR_420);
acldvppSetPicDescWidth(vpcInputDesc_, inputWidth_);
acldvppSetPicDescHeight(vpcInputDesc_, inputHeight_);
acldvppSetPicDescWidthStride(vpcInputDesc_, jpegOutWidthStride);
acldvppSetPicDescHeightStride(vpcInputDesc_, jpegOutHeightStride);
acldvppSetPicDescSize(vpcInputDesc_, jpegOutBufferSize);

// 7. Create the description of the output images and set the attribute values.
// If the cropped and pasted output image is used as the input for model inference, the output width and height must meet the requirements of the model.
// vpcOutputDesc_ is of type acldvppPicDesc.
vpcOutputBatchDesc_ = acldvppCreateBatchPicDesc(2);
for (uint32_t index=0; index<2; ++index){
     vecOutPtr_.push_back(vpcOutBufferDev_);
     vpcOutputDesc_ = acldvppGetPicDesc(vpcInputBatchDesc_, index);
    acldvppSetPicDescData(vpcOutputDesc_, vpcOutBufferDev_);
    acldvppSetPicDescFormat(vpcOutputDesc_, PIXEL_FORMAT_YUV_SEMIPLANAR_420);
    acldvppSetPicDescWidth(vpcOutputDesc_, dvppOutWidth);
    acldvppSetPicDescHeight(vpcOutputDesc_, dvppOutHeight);
    acldvppSetPicDescWidthStride(vpcOutputDesc_, dvppOutWidthStride);
    acldvppSetPicDescHeightStride(vpcOutputDesc_, dvppOutHeightStride);
    acldvppSetPicDescSize(vpcOutputDesc_, vpcOutBufferSize_);
}

// 8. Create roiNums. Each image corresponds to the number of cropped and pasted images.

uint32_ttotalNum = 0;
std::unique_ptr<uint32_t[]> roiNums(new (std::nothrow) uint32_t[1]);
roiNums[0]=2;
// 11. Perform asynchronous cropping and pasting, and call aclrtSynchronizeStream to wait for the stream tasks to complete.
ret = acldvppVpcBatchCropAndPasteAsync(dvppChannelDesc_, vpcInputBatchDesc_, roiNums.get(), 1,
        vpcOutputBatchDesc_, cropAreas_, pasteAreas_, stream_);
ret = aclrtSynchronizeStream(stream_);

// 9. After the cropping and pasting are complete, release the resources, including the description of the input and output images, input and output buffers, channel description, and channel.
acldvppDestroyRoiConfig(cropAreas_[0]);
acldvppDestroyRoiConfig(cropAreas_[1]);
acldvppDestroyRoiConfig(pasteAreas_[0]);
acldvppDestroyRoiConfig(pasteAreas_[1]);
(void)acldvppFree(vpcInDevBuffer_);
for(uint32_t index=0; index<2; ++index){
if(runMode == ACL_HOST){ 
    // In this mode, the processing result is on the device. Therefore, you need to call the memory copy API to transfer the result data, and then free the device buffer.
    // Allocate host buffer vpcOutHostBuffer.
    void* vpcOutHostBuffer = nullptr;
    vpcOutHostBuffer = malloc(vpcOutBufferSize_);
    // Transfer the processing result of the device to the host by using the aclrtMemcpy call.
    ret = aclrtMemcpy(vpcOutHostBuffer, vpcOutBufferSize_, vpcOutBufferDev_, vpcOutBufferSize_, ACL_MEMCPY_DEVICE_TO_HOST);
    // Free the input and output buffer on the device.
    (void)acldvppFree(vpcOutBufferDev_);
    // Free the buffer after the data is used.
    free(vpcOutHostBuffer);
} else { 
    // The process is running on the device, and the processing result is also on the device. Free the device buffer when the data is no longer needed.
    
    (void)acldvppFree(vpcOutBufferDev_);
}
}
acldvppDestroyBatchPicDesc(vpcInputDesc_);
acldvppDestroyBatchPicDesc(vpcOutputDesc_);
acldvppDestroyChannel(dvppChannelDesc_);
(void)acldvppDestroyChannelDesc(dvppChannelDesc_);
dvppChannelDesc_ = nullptr;

....
Parent topic: Media data processing V1
This site uses cookies to improve your browsing experience. By continuing to browse this site, you agree to our use of cookies.View Details