VDEC

This section describes the API call sequence of VDEC, and sample code is also provided to help you better understand the process.

The video decoder (VDEC) decodes H.264/H.265 video streams into YUV/RGB images. For details about the VDEC functions and restrictions, see aclvdecDestroyChannel.

API Call Sequence

If video decoding is involved during app development, ensure that your app contains the code logic for such video decoding. For details about the API call sequence, see AscendCL API Call Sequence.

Figure 1 Video decoding

This module implements video decoding. The key APIs are described as follows:

  1. Call aclvdecCreateChannel to create a video decoding channel.
    • Perform the following steps before creating a video decoding channel.
      1. Call aclvdecCreateChannelDesc to create channel description.
      2. Call the aclvdecSetChannelDesc series to set the decoding channel description, including the channel ID, thread, callback function, and video coding protocol.
        1. The callback function needs to be created by the user in advance. It is used to obtain the decoding output and destroy allocations in a timely manner. For details about the callback function prototype, see VPC.

          Call acldvppGetPicDescRetCode in the callback function to obtain the return code retCode. The value 0 indicates the success of decoding, while 1 indicates failure. If the decoding fails, locate the fault based on the return code in the log. For details, see Return Codes.

          After the decoding is complete, free the input and output buffers and destroy the stream description and image description in the callback function in a timely manner.

        2. The user needs to create a thread and define a thread function in advance. Once aclrtProcessReport is called in the thread function, the created callback function in 1.b.i will be triggered after a specified period of time.

        If aclvdecSetChannelDescOutPicFormat is not called to set the output format, images in YUV420SP (NV12) are output by default.

    • The following APIs are encapsulated in aclvdecCreateChannel and do not need to be called separately:
      1. aclrtCreateStream: explicitly creates a stream. It is internally used by VDEC.
      2. aclrtSubscribeReport: subscribes to a thread for handling the callback function in a stream. The callback function and thread are specified by the calls to the aclvdecSetChannelDesc series.
  2. Call aclvdecSendFrame to decode the video stream into YUV420SP images.
    • Perform the following steps before decoding a video stream:
    • During video decoding:

      aclrtLaunchCallback is encapsulated in aclvdecSendFrame to insert a callback function that needs to be executed to the stream. aclrtLaunchCallback does not need to be called separately.

    • After video decoding, use the callback function to obtain the result:

      But before that, obtain the value of retCode. The value 0 indicates the success of decoding, while 1 indicates failure. If the decoding fails, locate the fault based on the return code in the log. For details, see Return Codes.

      To obtain the frame sequence number returned after decoding, define the userData parameter of aclvdecSendFrame. Then, the sequence number can be passed to the VDEC callback function to determine the frame to be processed.

      To skip a specific frame, call aclvdecSendSkippedFrame to feed the frame (the input buffer) to the decoder. In this case, nullptr, instead of the decoding result, is output by the callback function after the decoding is complete.

  3. Call aclvdecDestroyChannel to destroy the channel.
    • The channel is destroyed only after the transmitted frames are decoded and the callback function is processed.
    • The following APIs are encapsulated in aclvdecDestroyChannel and do not need to be called separately:
    • After destroying the channel, call aclvdecDestroyChannelDesc to destroy the channel description.
    • After destroying the channel description, destroy the thread created in 1.b.ii.

Sample Code

You can view the complete code in Image Classification with ResNet-50 (Video Decoding+Synchronous Inference).

This section focuses on the code logic for decoding VDEC videos. For details about how to initialize and deinitialize AscendCL, see Initializing AscendCL. For details about how to allocate and deallocate runtime resources, see Runtime Resource Allocation and Deallocation.

After APIs are called, you need to add exception handling branches and record error logs and info logs. The following is a code snippet of key steps only, which is not ready to be built or run.

  1
  2
  3
  4
  5
  6
  7
  8
  9
 10
 11
 12
 13
 14
 15
 16
 17
 18
 19
 20
 21
 22
 23
 24
 25
 26
 27
 28
 29
 30
 31
 32
 33
 34
 35
 36
 37
 38
 39
 40
 41
 42
 43
 44
 45
 46
 47
 48
 49
 50
 51
 52
 53
 54
 55
 56
 57
 58
 59
 60
 61
 62
 63
 64
 65
 66
 67
 68
 69
 70
 71
 72
 73
 74
 75
 76
 77
 78
 79
 80
 81
 82
 83
 84
 85
 86
 87
 88
 89
 90
 91
 92
 93
 94
 95
 96
 97
 98
 99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
//1. Initialize AscendCL.

// 2. Allocate runtime resources.

// 3. Create a callback function.
void callback(acldvppStreamDesc *input, acldvppPicDesc *output, void *userdata)
{
    static int count = 1;
    if (output != nullptr) {
        // Obtain the VDEC output buffer, call the user-defined function WriteToFile to write output buffer data to a file, and then call acldvppFree to free the output buffer.
        void *vdecOutBufferDev = acldvppGetPicDescData(output);
        if (vdecOutBufferDev != nullptr) {
            // 0: vdec success; others, vdec failed
            // If retCode is 0, the decoding is successful. If retCode is 1, the decoding fails.
            int retCode = acldvppGetPicDescRetCode(output);
            if (retCode == 0) {
                // process task: write file
                uint32_t size = acldvppGetPicDescSize(output);
                std::string fileNameSave = "outdir/image" + std::to_string(count);
                // The VDEC output result is stored on the device. Perform the following operations in the WriteToFile method:
                if (!Utils::WriteToFile(fileNameSave.c_str(), vdecOutBufferDev, size)) {
                    ERROR_LOG("write file failed.");
                }
            } else {
                ERROR_LOG("vdec decode frame failed.");
            }

            // free output vdecOutBufferDev
            aclError ret = acldvppFree(vdecOutBufferDev);
        }
        // Destroy data of type acldvppPicDesc, which indicates the description data of the decoded output image.
        aclError ret = acldvppDestroyPicDesc(output);
    }

    // free input vdecInBufferDev and destroy stream desc
    if (input != nullptr) {
        void *vdecInBufferDev = acldvppGetStreamDescData(input);
        if (vdecInBufferDev != nullptr) {
            aclError ret = acldvppFree(vdecInBufferDev);
        }
         // Destroy data of type acldvppStreamDesc, which indicates the description data of the decoded input stream.
        aclError ret = acldvppDestroyStreamDesc(input);
    }

    INFO_LOG("success to callback %d.", count);
    count++;
}

//4. Create the description of the video stream processing channel, and set the attributes of the video processing channel description. The thread (threadId indicates the thread ID) and callback function need to be created in advance.
// vdecChannelDescis of type aclvdecChannelDesc.
vdecChannelDesc = aclvdecCreateChannelDesc();
aclError ret = aclvdecSetChannelDescChannelId(vdecChannelDesc, 10);
ret = aclvdecSetChannelDescThreadId(vdecChannelDesc, threadId);
ret = aclvdecSetChannelDescCallback(vdecChannelDesc, callback);
// In the sample, the H265_MAIN_LEVEL video coding protocol is used.
ret = aclvdecSetChannelDescEnType(vdecChannelDesc, static_cast<acldvppStreamFormat>(H265_MAIN_LEVEL));
// In the sample, PIXEL_FORMAT_YVU_SEMIPLANAR_420 is used.
ret = aclvdecSetChannelDescOutPicFormat(vdecChannelDesc, static_cast<acldvppPixelFormat>(PIXEL_FORMAT_YUV_SEMIPLANAR_420));

// 5. Create a channel for processing the video stream.
ret = aclvdecCreateChannel(vdecChannelDesc);


// 6. Call aclrtGetRunMode to obtain the run mode of the software stack. If ACL_DEVICE is returned, allocate and use the device buffer. If ACL_HOST is returned, transfer the input image data to the device by using the aclrtMemcpy call. After the data transfer is complete, the buffer needs to be freed in a timely manner.
aclrtRunMode runMode;
ret = aclrtGetRunMode(&runMode);

// 7. Send the stream in frame mode and decode the stream.
count = 0;
while (count < 10) { // Assume that there are 10 frames in the stream, so perform the send action 10 times.
// 7.1 Allocate input buffer (run mode specific) and read a frame of stream data. inBufferSize is the size of a frame to be read. You need to assign a value based on the actual stream size of each frame.
    if (runMode == ACL_HOST){ 
        // Allocate the host buffer inputHostBuff.
        void* inputHostBuff= nullptr;
    // inBufferSize is the size of the input stream.
        inputHostBuff= malloc(inBufferSize);
        // Load a frame of the input stream into the buffer. The ReadStreamFile function is defined by the user.
        ReadStreamFile(picName, inputHostBuff, inBufferSize);
        // Allocate the device buffer inBufferDev.
        ret = acldvppMalloc(&inBufferDev, inBufferSize);
        // Transfer the input image data to the device by using the aclrtMemcpy call.
        ret = aclrtMemcpy(inBufferDev, inBufferSize, inputHostBuff, inBufferSize, ACL_MEMCPY_HOST_TO_DEVICE);
        // Free the host buffer in a timely manner after data transfer is complete.
        free(inputHostBuff);
    } else {
        // Allocate device input buffer dataDev. inBufferSize is the size of each frame of the input stream.
        ret = acldvppMalloc(&inBufferDev, inBufferSize);
        // Load a frame of the input stream into the buffer. The ReadStreamFile function is defined by the user.
        ReadStreamFile(picName, inBufferDev, inBufferSize);
    }

    // 7.2 Create the description of the input video stream and set the stream attributes.
    streamInputDesc = acldvppCreateStreamDesc(); 
    // inBufferDev indicates the buffer for storing the input video data on the device, and inBufferSize indicates the buffer size.
    ret = acldvppSetStreamDescData(streamInputDesc, inBufferDev);
    ret = acldvppSetStreamDescSize(streamInputDesc, inBufferSize);

    // 7.3 Allocate device buffer picOutBufferDev for storing the VDEC-decoded output data. size is the size of the decoded image. Calculate the size based on the image format and the width and height of the aligned image, and then assign it to size.
    ret = acldvppMalloc(&picOutBufferDev, size);

    // 7.4 Create the description of the output image and set the attributes of the image description.
    // picOutputDesc is of type acldvppPicDesc
    picOutputDesc = acldvppCreatePicDesc();
    ret = acldvppSetPicDescData(picOutputDesc, picOutBufferDev);
    ret = acldvppSetPicDescSize(picOutputDesc, size);
    ret = acldvppSetPicDescFormat(picOutputDesc, static_cast<acldvppPixelFormat>(PIXEL_FORMAT_YUV_SEMIPLANAR_420));

    // 7.5 Decode the video stream. The callback function is called after each decoded frame to write data to the file, and destroy allocations in a timely manner.
    ret = aclvdecSendFrame(vdecChannelDesc, streamInputDesc, picOutputDesc, nullptr, nullptr);
    // ......
    count++;
// Note that aclvdecSendFrame is an asynchronous API. The input buffer (inBufferDev), output buffer (picOutBufferDev), input descriptor (streamInputDesc), and output descriptor (picOutputDesc) cannot be freed in advance and need to be freed in the callback function.
}

// 8. Destroy the image processing channel and image description information.
ret = aclvdecDestroyChannel(vdecChannelDesc);
aclvdecDestroyChannelDesc(vdecChannelDesc);

// 9. Deallocate runtime resources.

// 10. Deinitialize AscendCL.

// ......

Return Codes

Table 1 Return codes

Return Code

Description

Solution

AICPU_DVPP_KERNEL_STATE_SUCCESS = 0

Decoding success

-

AICPU_DVPP_KERNEL_STATE_FAILED = 1

Other error

-

AICPU_DVPP_KERNEL_STATE_DVPP_ERROR = 2

Failed to call APIs of other modules

-

AICPU_DVPP_KERNEL_STATE_PARAM_INVALID = 3

Argument verification failure

Check that the API arguments meet the API requirements.

AICPU_DVPP_KERNEL_STATE_OUTPUT_SIZE_INVALID = 4

Output buffer size verification failure

Check that the output buffer size meets the API requirements.

AICPU_DVPP_KERNEL_STATE_INTERNAL_ERROR = 5

System internal error

-

AICPU_DVPP_KERNEL_STATE_QUEUE_FULL = 6

Full internal queue

-

AICPU_DVPP_KERNEL_STATE_QUEUE_EMPTY = 7

Empty internal queue

-

AICPU_DVPP_KERNEL_STATE_QUEUE_NOT_EXIST = 8

Nonexistent internal queue

-

AICPU_DVPP_KERNEL_STATE_GET_CONTEX_FAILED = 9

Internal context obtaining failure

-

AICPU_DVPP_KERNEL_STATE_SUBMIT_EVENT_FAILED = 10

Internal event submission failure

-

AICPU_DVPP_KERNEL_STATE_MEMORY_FAILED = 11

Internal memory allocation failure

Check that the system has available memory.

AICPU_DVPP_KERNEL_STATE_SEND_NOTIFY_FAILED = 12

Internal notification push failure

-

AICPU_DVPP_KERNEL_STATE_VPC_OPERATE_FAILED = 13

Internal API operation failure

-

AICPU_DVPP_KERNEL_STATE_CHANNEL_ABNORMAL = 14

Abnormal channel

-

ERR_INVALID_STATE = 0x10001

Abnormal VDEC decoder state

-

ERR_HARDWARE = 0x10002

Hardware error, including decoder starts, execution, and stops

-

ERR_SCD_CUT_FAIL = 0x10003

An error occurs when the video stream is divided into multi-frame images.

Check that the input video stream is valid.

ERR_VDM_DECODE_FAIL = 0x10004

Single-frame decoding failure

Check that the input video stream is valid.

ERR_ALLOC_MEM_FAIL = 0x10005

Internal memory allocation failure

Check whether the system has available memory. If this error is ignored, no decoding result may be output even with stream input.

ERR_ALLOC_DYNAMIC_MEM_FAIL = 0x10006

Out-of-range input video resolution and dynamic memory allocation failure

Check the resolution of the input video stream and whether the system has available memory. If this error is ignored, no decoding result may be output with the continuously sent streams.

ERR_ALLOC_IN_OR_OUT_PORT_MEM_FAIL = 0x10007

Internal VDEC input and output buffer allocation failure

Check whether the system has available memory. If this error is ignored, no decoding result may be output even with stream input.

ERR_BITSTREAM = 0x10008

Invalid stream (for example, syntax error, eos retried, or eos detected in the first frame)

Check that the input video stream is valid.

ERR_VIDEO_FORMAT = 0x10009

Incorrect input video format

Check that the input video format is H.264 or H.265.

ERR_IMAGE_FORMAT = 0x1000a

Incorrect output format

Check that the output image format is NV12 or NV21.

ERR_CALLBACK = 0x1000b

Empty callback function

Check whether the callback function is empty.

ERR_INPUT_BUFFER = 0x1000c

Empty input buffer

Check whether the input buffer is empty.

ERR_INBUF_SIZE = 0x1000d

Input buffer size ≤ 0

Check whether the buffer size is less than or equal to 0.

ERR_THREAD_CREATE_FBD_FAIL = 0x1000e

Abnormal thread (thread where the callback function returns the decoding result)

Check that the resources (such as threads and memory) in the system are available.

ERR_CREATE_INSTANCE_FAIL = 0x1000f

Decoding instance creation failure

-

ERR_INIT_DECODER_FAIL = 0x10010

Decoder initialization failure. For example, the number of decoding instances exceeds the maximum of 16.

-

ERR_GET_CHANNEL_HANDLE_FAIL = 0x10011

Channel handle obtaining failure

-

ERR_COMPONENT_SET_FAIL = 0x10012

Decoding instance setting failure

Check that the input arguments are correct, for example, the input video format video_format and output frame format image_format.

ERR_COMPARE_NAME_FAIL = 0x10013

Decoding instance naming failure

Check that the input arguments are correct, for example, the input video format video_format and output frame format image_format.

ERR_OTHER = 0x10014

Other error

-

ERR_DECODE_NOPIC = 0x20000

Can be ignored. In the interlaced scenario where two fields are transmitted every frame, one of every two transmitted fields has no decoding output. This is expected. The decoding output of an interlaced stream is stored in the output buffers of the odd fields.

-

0x20001

Reference frame count setting failure

Check that the actual number of reference frames of the stream is the same as the configured number of reference frames.

0x20002

Size of VDEC frame buffer setting failure

Check that the actual width and height of the input stream are the same as the configured width and height.

0xA0058001

Invalid device ID

Check the device ID.

0xA0058002

Invalid channel ID

Check that the input channel ID is correct or whether the total number of channels reaches the upper limit.

0xA0058003

Invalid argument, such as an invalid enum value

Check the incorrect argument based on the logs.

0xA0058004

Existing resource

Check whether the channel has been repeatedly created.

0xA0058005

Non-existent channel resource

Check whether a non-existent channel ID or channel handle is used.

0xA0058006

Null pointer

Check the input of the API based on the logs.

0xA0058007

No configuration made to corresponding parameters before the system, device, or channel is enabled.

Check the input of the API based on the logs.

0xA0058008

Unsupported argument or function.

Check the input of the API based on the logs.

0xA0058009

Unallowed operation, for example, an attempt to update a static attribute.

-

0xA005800C

Memory allocation failure due to memory insufficiency or other reasons.

Check whether the system has available memory. If this error is ignored, no decoding result may be output even with stream input.

0xA005800D

Buffer allocation failure due to too large requested picture buffer or other reasons

-

0xA005800E

Empty buffer

Decoding is not complete, so there is no decoding result in the buffer. Try again after the decoding result is loaded to the buffer.

0xA005800F

Full buffer

Stream feeding is too fast, which causes full input buffer. Slow down stream feeding or increase the buffer allocation when creating a channel.

0xA0058010

Uninitialized system or unloaded dependent modules

Check that the system initialization API is called.

0xA0058011

Bad address

-

0xA0058012

Busy system

Check whether the total number of VDEC channels reaches the upper limit.

0xA0058013

Not enough buffer

-

0xA0058014

Hardware or software processing timeout

-

0xA0058015

System error

-

0xA005803F

Maximum return. The error code value of the module must be less than this value.

-