VDEC

This section describes the API call sequence of VDEC, and sample code is also provided to help you better understand the process.

The video decoder (VDEC) decodes H.264/H.265 video streams into YUV/RGB images. For details about the VDEC function and restrictions, see DVPP Media Acceleration Library.

API Call Sequence

Figure 1 Video decoding workflow

This module implements video decoding. The key APIs are described as follows:

  1. Call aclvdecCreateChannel to create a channel for video decoding.
    • Perform the following steps before creating a video decoding channel.
      1. Call aclvdecCreateChannelDesc to create channel description.
      2. Call aclvdecSetChannelDesc series APIs to set attributes of the channel description, including the decoding channel ID, thread, callback function, and video encoding protocol.
        1. The callback function needs to be created in advance. It is used to obtain the decoding data after video decoding and free resources in time. For details about the callback function prototype, see <cf id="Bold">aclvdecCallback</cf>.

          Call <cf id="Bold">acldvppGetPicDescRetCode</cf> in the callback function to obtain the return code <cf id="Bold">retCode</cf>. The value <cf id="Bold">0</cf> indicates the success of decoding, while <cf id="Bold">1</cf> indicates failure. If the decoding fails, locate the fault based on the return code in the log. For details, see Return Code.

          After the decoding is complete, free the input and output buffers and destroy the stream description and image description in the callback function in a timely manner.

        2. The user needs to create a thread in advance and customize a thread function. When aclrtProcessReport is called in the thread function, the callback function in 1.b.i is triggered after a specified period of time.

        If aclvdecSetChannelDescOutPicFormat is not called to set the output format, YUV420SP NV12 is used by default.

    • The following APIs are encapsulated in aclvdecCreateChannel and do not need to be called separately:
      1. <cf id="Bold">aclrtCreateStream</cf>: explicitly creates a stream. It is used internally by VDEC.
      2. aclrtSubscribeReport: specifies the thread for processing the callback function in a stream. The callback function and thread are specified by calling the aclvdecSetChannelDesc series APIs.
  2. Call aclvdecSendFrame to decode the video stream into a YUV420SP image.
    • Perform the following steps before decoding a video stream:
      • Call <cf id="Bold">acldvppCreateStreamDesc</cf> to create the description of the input video stream, and call <cf id="Bold">acldvppSetStreamDesc</cf> series to configure the input video, including the memory address, memory size, and stream format.
      • Call <cf id="Bold">acldvppCreatePicDesc</cf> to create the description of the output image, and call <cf id="Bold">acldvppSetPicDesc</cf> series to configure the output image, including the memory address, memory size, and image format.
    • During video decoding:

      aclrtLaunchCallback is encapsulated in aclvdecSendFrame to add a callback function to be executed to the stream task queue. Therefore, <cf id="Bold">aclrtLaunchCallback</cf> does not need to be called separately.

    • After video decoding, use the callback function to obtain the result:

      But before that, obtain the value of retCode. The value 0 indicates the success of decoding, while 1 indicates failure. If the decoding fails, locate the fault based on the return code in the log. For details, see Return Code.

      To obtain the sequence number of the decoded frame, define the userData parameter of the aclvdecSendFrame interface. The sequence number of the decoded frame can be transferred to the VDEC callback function through the userData parameter to determine the sequence number of the frame to be processed in the callback function.

      If you do not want to obtain the decoding result of a frame, call aclvdecSendSkippedFrame to transfer the stream (input memory) to be decoded to the decoder for decoding. In this case, the decoding result is not output. The output returned by the decoding callback function is nullptr.

  3. Call aclvdecDestroyChannel to destroy the video processing channel.
    • The channel is destroyed only after the transmitted frames are decoded and the callback function is processed.
    • The following APIs are encapsulated in aclvdecDestroyChannel and do not need to be called separately:
      • aclrtUnSubscribeReport: Deregisters a thread. (The callback function in the stream is no longer processed by the specified thread.)
      • <cf id="Bold">aclrtDestroyStream</cf>: destroys a stream.
    • After destroying the channel, call <cf id="Bold">aclvdecDestroyChannelDesc</cf> to destroy the channel description.
    • You can destroy the thread created in 1.b.ii only after destroying the channel description.

Sample Code

The following is a code example of key steps of the VDEC video decoding function. It is for reference only and cannot be directly copied for compilation and running. After APIs are called, you need to add exception handling branches and record error logs and info logs.

You can click vdec_resnet50_classification to obtain the sample.

  1
  2
  3
  4
  5
  6
  7
  8
  9
 10
 11
 12
 13
 14
 15
 16
 17
 18
 19
 20
 21
 22
 23
 24
 25
 26
 27
 28
 29
 30
 31
 32
 33
 34
 35
 36
 37
 38
 39
 40
 41
 42
 43
 44
 45
 46
 47
 48
 49
 50
 51
 52
 53
 54
 55
 56
 57
 58
 59
 60
 61
 62
 63
 64
 65
 66
 67
 68
 69
 70
 71
 72
 73
 74
 75
 76
 77
 78
 79
 80
 81
 82
 83
 84
 85
 86
 87
 88
 89
 90
 91
 92
 93
 94
 95
 96
 97
 98
 99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
// 1. Create a callback function.
void callback(acldvppStreamDesc *input, acldvppPicDesc *output, void *userdata)
{
    static int count = 1;
    if (output != nullptr) {
        //Obtain the VDEC output buffer, call the user-defined function WriteToFile to write output buffer data to a file, and then call acldvppFree to free the output buffer.
        void *vdecOutBufferDev = acldvppGetPicDescData(output);
        if (vdecOutBufferDev != nullptr) {
            // 0: vdec success; others, vdec failed
            // If retCode is 0, the decoding is successful. If retCode is 1, the decoding fails.
            int retCode = acldvppGetPicDescRetCode(output);
            if (retCode == 0) {
                // process task: write file
                uint32_t size = acldvppGetPicDescSize(output);
                std::string fileNameSave = "outdir/image" + std::to_string(count);
                // VDEC output result is stored on the device. Perform the following operations in the WriteToFile method:
                if (!Utils::WriteToFile(fileNameSave.c_str(), vdecOutBufferDev, size)) {
                    ERROR_LOG("write file failed.");
                }
            } else {
                ERROR_LOG("vdec decode frame failed.");
            }

            // free output vdecOutBufferDev
            aclError ret = acldvppFree(vdecOutBufferDev);
        }
        //Destroy data of type acldvppPicDesc, that is, the output image description.
        aclError ret = acldvppDestroyPicDesc(output);
    }

    // free input vdecInBufferDev and destroy stream desc
    if (input != nullptr) {
        void *vdecInBufferDev = acldvppGetStreamDescData(input);
        if (vdecInBufferDev != nullptr) {
            aclError ret = acldvppFree(vdecInBufferDev);
        }
        //Destroy data of type acldvppStreamDesc,  that is, the input stream description.
        aclError ret = acldvppDestroyStreamDesc(input);
    }

    INFO_LOG("success to callback %d.", count);
    count++;
}

// 2. Set the attributes of the video stream processing channel description. The thread (threadId indicates the thread ID) and callback function need to be created in advance.
// vdecChannelDesc is of type aclvdecChannelDesc.
vdecChannelDesc = aclvdecCreateChannelDesc();
aclError ret = aclvdecSetChannelDescChannelId(vdecChannelDesc, 10);
ret = aclvdecSetChannelDescThreadId(vdecChannelDesc, threadId);
ret = aclvdecSetChannelDescCallback(vdecChannelDesc, callback);
// In the sample, the H265_MAIN_LEVEL video coding protocol is used.
ret = aclvdecSetChannelDescEnType(vdecChannelDesc, static_cast<acldvppStreamFormat>(H265_MAIN_LEVEL));
// In the sample, PIXEL_FORMAT_YVU_SEMIPLANAR_420 is used.
ret = aclvdecSetChannelDescOutPicFormat(vdecChannelDesc, static_cast<acldvppPixelFormat>(PIXEL_FORMAT_YUV_SEMIPLANAR_420));

// 3. Create a channel for processing video streams.
ret = aclvdecCreateChannel(vdecChannelDesc);


// 4. Call aclrtGetRunMode to obtain the run mode of the software stack. If the run mode obtained by calling aclrtGetRunMode is ACL_HOST, call aclrtMemcpy to transfer the input image data to the device. After the data transfer is complete, free the memory in a timely manner. Otherwise, the memory of the device is directly allocated and used.
aclrtRunMode runMode;
ret = aclrtGetRunMode(&runMode);

// 5. Send the stream by frame and decode it.
count = 0;
while (count < 10) { // Assume that there are 10 frames in the stream, so perform the send action 10 times.
// 5.1 Apply for the input stream memory (differentiated by the running status) and read a frame of stream data. inBufferSize is the size of a frame to be read. You need to assign a value based on the actual stream size of each frame.
    if (runMode == ACL_HOST){ 
        // Allocate the host buffer inputHostBuff.
        void* inputHostBuff= nullptr;
        // inBufferSize is the size of the input stream.
        inputHostBuff= malloc(inBufferSize);
        // Load a frame of the input stream into the buffer. The ReadStreamFile function is defined by the user.
        ReadStreamFile(picName, inputHostBuff, inBufferSize);
        // Allocate the device buffer inBufferDev.
        ret = acldvppMalloc(&inBufferDev, inBufferSize);
    // Transfer the input image data to the device by using the aclrtMemcpy call.
        ret = aclrtMemcpy(inBufferDev, inBufferSize, inputHostBuff, inBufferSize, ACL_MEMCPY_HOST_TO_DEVICE);
        // Free the host buffer in a timely manner after data transfer is complete.
        free(inputHostBuff);
    } else {
        // Allocate device input buffer dataDev. inBufferSize is the size of each frame of the input stream.
        ret = acldvppMalloc(&inBufferDev, inBufferSize);
        // Load a frame of the input stream into the buffer. The ReadStreamFile function is defined by the user.
        ReadStreamFile(picName, inBufferDev, inBufferSize);
    }

// 5.2 Create the description of the input video stream and set the attributes of the stream information.
    streamInputDesc = acldvppCreateStreamDesc(); 
    // inBufferDev indicates the buffer for storing the input video data on the device, and inBufferSize indicates the buffer size.
    ret = acldvppSetStreamDescData(streamInputDesc, inBufferDev);
    ret = acldvppSetStreamDescSize(streamInputDesc, inBufferSize);

// 5.3 Allocate the device buffer picOutBufferDev for storing the output data after VDEC decoding. The size is the size of the decoded image. You need to calculate the size based on the image format and the width and height of the aligned image, and then assign a value to the size.
    ret = acldvppMalloc(&picOutBufferDev, size);

// 5.4 Create the description of the output image and set the attributes of the image description.
    // picOutputDesc is of type acldvppPicDesc
    picOutputDesc = acldvppCreatePicDesc();
    ret = acldvppSetPicDescData(picOutputDesc, picOutBufferDev);
    ret = acldvppSetPicDescSize(picOutputDesc, size);
    ret = acldvppSetPicDescFormat(picOutputDesc, static_cast<acldvppPixelFormat>(PIXEL_FORMAT_YUV_SEMIPLANAR_420));

// 5.5 Decode video streams. After each frame of data is decoded, the system automatically calls the callback function to write the decoded data to a file, and then releases related resources in a timely manner.
    ret = aclvdecSendFrame(vdecChannelDesc, streamInputDesc, picOutputDesc, nullptr, nullptr);
    // ......
    count++;
    // Note that aclvdecSendFrame is an asynchronous API. The input buffer (inBufferDev), output buffer (picOutBufferDev), input descriptor (streamInputDesc), and output descriptor (picOutputDesc) cannot be freed in advance and need to be freed in the callback function.
}

// 6. Release the image processing channel and image description.
ret = aclvdecDestroyChannel(vdecChannelDesc);
aclvdecDestroyChannelDesc(vdecChannelDesc);

......

Return Code

Table 1 Return code

Return Code

Description

Possible Cause and Solution

AICPU_DVPP_KERNEL_STATE_SUCCESS = 0

Decoding success

-

AICPU_DVPP_KERNEL_STATE_FAILED = 1

Other errors

-

AICPU_DVPP_KERNEL_STATE_DVPP_ERROR = 2

Failure of calling APIs of other modules

-

AICPU_DVPP_KERNEL_STATE_PARAM_INVALID = 3

Parameter verification failed.

Check that the API arguments meet the API requirements.

AICPU_DVPP_KERNEL_STATE_OUTPUT_SIZE_INVALID = 4

Output buffer size verification failure

Check that the output buffer size meets the API requirements.

AICPU_DVPP_KERNEL_STATE_INTERNAL_ERROR = 5

System internal error

-

AICPU_DVPP_KERNEL_STATE_QUEUE_FULL = 6

Full internal queue

-

AICPU_DVPP_KERNEL_STATE_QUEUE_EMPTY = 7

Empty internal queue

-

AICPU_DVPP_KERNEL_STATE_QUEUE_NOT_EXIST = 8

Nonexistent internal queue

-

AICPU_DVPP_KERNEL_STATE_GET_CONTEX_FAILED = 9

Internal context obtaining failure

-

AICPU_DVPP_KERNEL_STATE_SUBMIT_EVENT_FAILED = 10

Internal event submission failure

-

AICPU_DVPP_KERNEL_STATE_MEMORY_FAILED = 11

Internal memory allocation failure

Check that the system has available memory.

AICPU_DVPP_KERNEL_STATE_SEND_NOTIFY_FAILED = 12

Internal notification push failure

-

AICPU_DVPP_KERNEL_STATE_VPC_OPERATE_FAILED = 13

Internal API operation failure

-

AICPU_DVPP_KERNEL_STATE_CHANNEL_ABNORMAL = 14

Abnormal channel

-

ERR_INVALID_STATE = 0x10001

Abnormal VDEC state

-

ERR_HARDWARE = 0x10002

Hardware error, including decoder starts, execution, and stops

-

ERR_SCD_CUT_FAIL = 0x10003

Abnormal video stream-to-frame cutting

Check that the input video stream is valid.

ERR_VDM_DECODE_FAIL = 0x10004

Single-frame decoding failure

Check that the input video stream data is correct.

ERR_ALLOC_MEM_FAIL = 0x10005

Internal memory allocation failure

Check whether the system has available memory. If this error is ignored, no decoding result may be output even with stream input.

ERR_ALLOC_DYNAMIC_MEM_FAIL = 0x10006

Out-of-range input video resolution and dynamic memory allocation failure

Check the resolution of the input video stream and that the system has available memory. If this error is ignored, no decode result may be output as more stream frames are fed to the decoder.

ERR_ALLOC_IN_OR_OUT_PORT_MEM_FAIL = 0x10007

Internal VDEC input and output buffer allocation failure

Check that the system has available memory. If this error is ignored, no decode result may be output as more stream frames are fed to the decoder.

ERR_BITSTREAM = 0x10008

Invalid stream (for example, syntax error, eos retried, or eos detected in the first frame)

Check that the input video stream data is correct.

ERR_VIDEO_FORMAT = 0x10009

Incorrect input video format

Check that the input video format is H.264 or H.265.

ERR_IMAGE_FORMAT = 0x1000a

Incorrect output format

Check that the output image format is NV12 or NV21.

ERR_CALLBACK = 0x1000b

Empty callback function

Check that the callback function is empty.

ERR_INPUT_BUFFER = 0x1000c

Empty input buffer

Check whether the input buffer is empty.

ERR_INBUF_SIZE = 0x1000d

Input buffer size ≤ 0

Check that the buffer size is less than or equal to 0.

ERR_THREAD_CREATE_FBD_FAIL = 0x1000e

Abnormal thread (thread where the callback function returns the decoding result)

Check that the resources (such as threads and memory) in the system are available.

ERR_CREATE_INSTANCE_FAIL = 0x1000f

Decoding instance creation failure

-

ERR_INIT_DECODER_FAIL = 0x10010

Decoder initialization failure. For example, the number of decoding instances exceeds the maximum of 16.

-

ERR_GET_CHANNEL_HANDLE_FAIL = 0x10011

Channel handle obtaining failure

-

ERR_COMPONENT_SET_FAIL = 0x10012

Decoding instance setting failure

Check whether the input arguments of VDEC are correct, such as the input video format (video_format) and output frame format (image_format).

ERR_COMPARE_NAME_FAIL = 0x10013

Decoding instance naming failure

Check that the input arguments are correct, for example, the input video format video_format and output frame format image_format.

ERR_OTHER = 0x10014

Other errors

-

ERR_DECODE_NOPIC = 0x20000

Can be ignored. In the interlaced scenario where two fields are transmitted every frame, one of every two transmitted fields has no decoding output. This is expected. The decoding output of an interlaced stream is stored in the output buffers of the odd fields.

-

0x20001

Reference frame count setting failure

Check that the actual number of reference frames of the stream is the same as the configured number of reference frames. The number of reference frames defaults to 8 for the Atlas inference product.

0x20002

Size of VDEC frame buffer setting failure

Check that the actual width and height of the input stream are the same as the configured width and height.

0xA0058001

Invalid device ID.

Check the device ID.

0xA0058002

Invalid channel ID

Check that the input channel ID is correct or whether the total number of channels reaches the upper limit.

0xA0058003

Invalid argument, such as an invalid enum value

Check the incorrect argument based on the logs.

0xA0058004

Existing resource

Check whether the channel has been repeatedly created.

0xA0058005

Non-existent channel resource

Check whether a non-existent channel ID or channel handle is used.

0xA0058006

Null pointer

Check the input argument of the API based on the logs.

0xA0058007

No configuration made to corresponding parameters before the system, device, or channel is enabled.

Check the input argument of the API based on the logs.

0xA0058008

Unsupported parameter or function.

Check the input argument of the API based on the logs.

0xA0058009

Unallowed operation, for example, modifying static configuration parameters

-

0xA005800C

Memory allocation failure due to system memory insufficiency or other reasons

Check that the system has available memory. If this error is ignored, no decode result may be output as more stream frames are fed to the decoder.

0xA005800D

Buffer allocation failure due to too large requested picture buffer or other reasons

-

0xA005800E

Empty buffer

Decoding is not complete, so there is no decode result in the buffer. Try again after the decode result is loaded to the buffer.

0xA005800F

Full buffer

Stream feeding is too fast, which causes full input buffer. Slow down stream feeding or increase the buffer allocation when creating a channel.

0xA0058010

Uninitialized system or unloaded dependent modules

Check that the system initialization API is called.

0xA0058011

Bad address

-

0xA0058012

Busy system

Check whether the total number of VDEC channels reaches the upper limit.

0xA0058013

Not enough buffer

-

0xA0058014

Hardware or software processing timeout

-

0xA0058015

System error

-

0xA005803F

Maximum return. The error code value of the module must be less than this value.

-