Choosing a Proper VDEC Output Format and Resolution to Optimize Performance

Background

On some Ascend AI Processors, the aclvdecSendFrame API supports YUV420SP or RGB888 output and allows image resizing during decoding. If you want to use DVPP to decode videos and obtain RGB images, the aclvdecSendFrame API can be called directly to output RGB888 data, thus improving performance.

Figure 1 VDEC output scenarios

Principles

On some Ascend AI Processors, the aclvdecSendFrame API supports YUV420SP or RGB888 output (for details, see Functions and Restrictions). You can set the API parameters to select different output formats. In this way, you do not need to call acldvppVpcConvertColorAsync to convert the format, reducing the API calling time.

If the resolution of a video stream is different from that of an image input to the model, the decoded image needs to be resized. You can also set the resolution of the output image by calling aclvdecSendFrame. In this way, the image can be resized during decoding, with no need for you to call the resizing API additionally.

In brief, the aclvdecSendFrame call can implement the functions of decoding, resizing, and CSC, reducing the number of APIs to be called and improving performance.

Sample

Refer to vdec_resnet50_classification to obtain the sample. to get the complete code of video decoding and inference.

To output RGB888 data for video decoding and implement resizing at the same time, set the image format, width, height, width stride, and height stride in the output parameter (indicating the output image description) of the aclvdecSendFrame API. The following is an example:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23

//Create output image description.
acldvppPicDesc picOutputDesc = acldvppCreatePicDesc();

//Define the output image format, width, height, width stride, height stride, and output buffer.
uint32_t width = 224;
uint32_t height = 224;
uint32_t widthStride = 224 * 3;
uint32_t heightStride = 224;
uint32_t size = widthStride * heightStride;
void *picOutBufferDev = nullptr;
acldvppMalloc(&picOutBufferDev, size);

//Set the output image format, width, height, width stride, height stride, and output buffer.
acldvppSetPicDescData(picOutputDesc, picOutBufferDev);
acldvppSetPicDescSize(picOutputDesc, size);
acldvppSetPicDescFormat(picOutputDesc, PIXEL_FORMAT_RGB_888);
acldvppSetPicDescWidth(picOutputDesc, width);
acldvppSetPicDescHeight(picOutputDesc, height);
acldvppSetPicDescWidthStride(picOutputDesc, widthStride);
acldvppSetPicDescHeightStride(picOutputDesc, heightStride);

//Call the decoding API.
aclvdecSendFrame(channelDesc, picInputDesc, picOutputDesc, nullptr, nullptr);