VDEC
This section describes the API call sequence of VDEC, and sample code is also provided to help you better understand the process.
The video decoder (VDEC) decodes H.264/H.265 video streams into YUV/RGB images. For details about the VDEC functions and restrictions, see aclvdecDestroyChannel.
API Call Sequence
If video decoding is involved during app development, ensure that your app contains the code logic for such video decoding. For details about the API call sequence, see AscendCL API Call Sequence.
This module implements video decoding. The key APIs are described as follows:
- Call aclvdecCreateChannel to create a video decoding channel.
- Perform the following steps before creating a video decoding channel.
- Call aclvdecCreateChannelDesc to create channel description.
- Call the aclvdecSetChannelDesc series to set the decoding channel description, including the channel ID, thread, callback function, and video coding protocol.
- The callback function needs to be created by the user in advance. It is used to obtain the decoding output and destroy allocations in a timely manner. For details about the callback function prototype, see VPC.
Call acldvppGetPicDescRetCode in the callback function to obtain the return code retCode. The value 0 indicates the success of decoding, while 1 indicates failure. If the decoding fails, locate the fault based on the return code in the log. For details, see Return Codes.
After the decoding is complete, free the input and output buffers and destroy the stream description and image description in the callback function in a timely manner.
- The user needs to create a thread and define a thread function in advance. Once aclrtProcessReport is called in the thread function, the created callback function in 1.b.i will be triggered after a specified period of time.
- The callback function needs to be created by the user in advance. It is used to obtain the decoding output and destroy allocations in a timely manner. For details about the callback function prototype, see VPC.
- The following APIs are encapsulated in aclvdecCreateChannel and do not need to be called separately:
- Perform the following steps before creating a video decoding channel.
- Call aclvdecSendFrame to decode the video stream into YUV420SP images.
- Perform the following steps before decoding a video stream:
- Call acldvppCreateStreamDesc to create an input video stream description, and call the acldvppSetStreamDesc series to set the description, including the buffer address, buffer size, and stream format.
- Call acldvppCreatePicDesc to create an output image description, and call the acldvppSetPicDesc series to set the description, including the buffer address, buffer size, and image format.
- During video decoding:
aclrtLaunchCallback is encapsulated in aclvdecSendFrame to insert a callback function that needs to be executed to the stream. aclrtLaunchCallback does not need to be called separately.
- After video decoding, use the callback function to obtain the result:
But before that, obtain the value of retCode. The value 0 indicates the success of decoding, while 1 indicates failure. If the decoding fails, locate the fault based on the return code in the log. For details, see Return Codes.
To obtain the frame sequence number returned after decoding, define the userData parameter of aclvdecSendFrame. Then, the sequence number can be passed to the VDEC callback function to determine the frame to be processed.
To skip a specific frame, call aclvdecSendSkippedFrame to feed the frame (the input buffer) to the decoder. In this case, nullptr, instead of the decoding result, is output by the callback function after the decoding is complete.
- Perform the following steps before decoding a video stream:
- Call aclvdecDestroyChannel to destroy the channel.
- The channel is destroyed only after the transmitted frames are decoded and the callback function is processed.
- The following APIs are encapsulated in aclvdecDestroyChannel and do not need to be called separately:
- After destroying the channel, call aclvdecDestroyChannelDesc to destroy the channel description.
- After destroying the channel description, destroy the thread created in 1.b.ii.
Sample Code
You can view the complete code in Image Classification with ResNet-50 (Video Decoding+Synchronous Inference).
This section focuses on the code logic for decoding VDEC videos. For details about how to initialize and deinitialize AscendCL, see Initializing AscendCL. For details about how to allocate and deallocate runtime resources, see Runtime Resource Allocation and Deallocation.
After APIs are called, you need to add exception handling branches and record error logs and info logs. The following is a code snippet of key steps only, which is not ready to be built or run.
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 |
//1. Initialize AscendCL. // 2. Allocate runtime resources. // 3. Create a callback function. void callback(acldvppStreamDesc *input, acldvppPicDesc *output, void *userdata) { static int count = 1; if (output != nullptr) { // Obtain the VDEC output buffer, call the user-defined function WriteToFile to write output buffer data to a file, and then call acldvppFree to free the output buffer. void *vdecOutBufferDev = acldvppGetPicDescData(output); if (vdecOutBufferDev != nullptr) { // 0: vdec success; others, vdec failed // If retCode is 0, the decoding is successful. If retCode is 1, the decoding fails. int retCode = acldvppGetPicDescRetCode(output); if (retCode == 0) { // process task: write file uint32_t size = acldvppGetPicDescSize(output); std::string fileNameSave = "outdir/image" + std::to_string(count); // The VDEC output result is stored on the device. Perform the following operations in the WriteToFile method: if (!Utils::WriteToFile(fileNameSave.c_str(), vdecOutBufferDev, size)) { ERROR_LOG("write file failed."); } } else { ERROR_LOG("vdec decode frame failed."); } // free output vdecOutBufferDev aclError ret = acldvppFree(vdecOutBufferDev); } // Destroy data of type acldvppPicDesc, which indicates the description data of the decoded output image. aclError ret = acldvppDestroyPicDesc(output); } // free input vdecInBufferDev and destroy stream desc if (input != nullptr) { void *vdecInBufferDev = acldvppGetStreamDescData(input); if (vdecInBufferDev != nullptr) { aclError ret = acldvppFree(vdecInBufferDev); } // Destroy data of type acldvppStreamDesc, which indicates the description data of the decoded input stream. aclError ret = acldvppDestroyStreamDesc(input); } INFO_LOG("success to callback %d.", count); count++; } //4. Create the description of the video stream processing channel, and set the attributes of the video processing channel description. The thread (threadId indicates the thread ID) and callback function need to be created in advance. // vdecChannelDescis of type aclvdecChannelDesc. vdecChannelDesc = aclvdecCreateChannelDesc(); aclError ret = aclvdecSetChannelDescChannelId(vdecChannelDesc, 10); ret = aclvdecSetChannelDescThreadId(vdecChannelDesc, threadId); ret = aclvdecSetChannelDescCallback(vdecChannelDesc, callback); // In the sample, the H265_MAIN_LEVEL video coding protocol is used. ret = aclvdecSetChannelDescEnType(vdecChannelDesc, static_cast<acldvppStreamFormat>(H265_MAIN_LEVEL)); // In the sample, PIXEL_FORMAT_YVU_SEMIPLANAR_420 is used. ret = aclvdecSetChannelDescOutPicFormat(vdecChannelDesc, static_cast<acldvppPixelFormat>(PIXEL_FORMAT_YUV_SEMIPLANAR_420)); // 5. Create a channel for processing the video stream. ret = aclvdecCreateChannel(vdecChannelDesc); // 6. Call aclrtGetRunMode to obtain the run mode of the software stack. If ACL_DEVICE is returned, allocate and use the device buffer. If ACL_HOST is returned, transfer the input image data to the device by using the aclrtMemcpy call. After the data transfer is complete, the buffer needs to be freed in a timely manner. aclrtRunMode runMode; ret = aclrtGetRunMode(&runMode); // 7. Send the stream in frame mode and decode the stream. count = 0; while (count < 10) { // Assume that there are 10 frames in the stream, so perform the send action 10 times. // 7.1 Allocate input buffer (run mode specific) and read a frame of stream data. inBufferSize is the size of a frame to be read. You need to assign a value based on the actual stream size of each frame. if (runMode == ACL_HOST){ // Allocate the host buffer inputHostBuff. void* inputHostBuff= nullptr; // inBufferSize is the size of the input stream. inputHostBuff= malloc(inBufferSize); // Load a frame of the input stream into the buffer. The ReadStreamFile function is defined by the user. ReadStreamFile(picName, inputHostBuff, inBufferSize); // Allocate the device buffer inBufferDev. ret = acldvppMalloc(&inBufferDev, inBufferSize); // Transfer the input image data to the device by using the aclrtMemcpy call. ret = aclrtMemcpy(inBufferDev, inBufferSize, inputHostBuff, inBufferSize, ACL_MEMCPY_HOST_TO_DEVICE); // Free the host buffer in a timely manner after data transfer is complete. free(inputHostBuff); } else { // Allocate device input buffer dataDev. inBufferSize is the size of each frame of the input stream. ret = acldvppMalloc(&inBufferDev, inBufferSize); // Load a frame of the input stream into the buffer. The ReadStreamFile function is defined by the user. ReadStreamFile(picName, inBufferDev, inBufferSize); } // 7.2 Create the description of the input video stream and set the stream attributes. streamInputDesc = acldvppCreateStreamDesc(); // inBufferDev indicates the buffer for storing the input video data on the device, and inBufferSize indicates the buffer size. ret = acldvppSetStreamDescData(streamInputDesc, inBufferDev); ret = acldvppSetStreamDescSize(streamInputDesc, inBufferSize); // 7.3 Allocate device buffer picOutBufferDev for storing the VDEC-decoded output data. size is the size of the decoded image. Calculate the size based on the image format and the width and height of the aligned image, and then assign it to size. ret = acldvppMalloc(&picOutBufferDev, size); // 7.4 Create the description of the output image and set the attributes of the image description. // picOutputDesc is of type acldvppPicDesc picOutputDesc = acldvppCreatePicDesc(); ret = acldvppSetPicDescData(picOutputDesc, picOutBufferDev); ret = acldvppSetPicDescSize(picOutputDesc, size); ret = acldvppSetPicDescFormat(picOutputDesc, static_cast<acldvppPixelFormat>(PIXEL_FORMAT_YUV_SEMIPLANAR_420)); // 7.5 Decode the video stream. The callback function is called after each decoded frame to write data to the file, and destroy allocations in a timely manner. ret = aclvdecSendFrame(vdecChannelDesc, streamInputDesc, picOutputDesc, nullptr, nullptr); // ...... count++; // Note that aclvdecSendFrame is an asynchronous API. The input buffer (inBufferDev), output buffer (picOutBufferDev), input descriptor (streamInputDesc), and output descriptor (picOutputDesc) cannot be freed in advance and need to be freed in the callback function. } // 8. Destroy the image processing channel and image description information. ret = aclvdecDestroyChannel(vdecChannelDesc); aclvdecDestroyChannelDesc(vdecChannelDesc); // 9. Deallocate runtime resources. // 10. Deinitialize AscendCL. // ...... |
Return Codes
|
Return Code |
Description |
Solution |
|---|---|---|
|
AICPU_DVPP_KERNEL_STATE_SUCCESS = 0 |
Decoding success |
- |
|
AICPU_DVPP_KERNEL_STATE_FAILED = 1 |
Other error |
- |
|
AICPU_DVPP_KERNEL_STATE_DVPP_ERROR = 2 |
Failed to call APIs of other modules |
- |
|
AICPU_DVPP_KERNEL_STATE_PARAM_INVALID = 3 |
Argument verification failure |
Check that the API arguments meet the API requirements. |
|
AICPU_DVPP_KERNEL_STATE_OUTPUT_SIZE_INVALID = 4 |
Output buffer size verification failure |
Check that the output buffer size meets the API requirements. |
|
AICPU_DVPP_KERNEL_STATE_INTERNAL_ERROR = 5 |
System internal error |
- |
|
AICPU_DVPP_KERNEL_STATE_QUEUE_FULL = 6 |
Full internal queue |
- |
|
AICPU_DVPP_KERNEL_STATE_QUEUE_EMPTY = 7 |
Empty internal queue |
- |
|
AICPU_DVPP_KERNEL_STATE_QUEUE_NOT_EXIST = 8 |
Nonexistent internal queue |
- |
|
AICPU_DVPP_KERNEL_STATE_GET_CONTEX_FAILED = 9 |
Internal context obtaining failure |
- |
|
AICPU_DVPP_KERNEL_STATE_SUBMIT_EVENT_FAILED = 10 |
Internal event submission failure |
- |
|
AICPU_DVPP_KERNEL_STATE_MEMORY_FAILED = 11 |
Internal memory allocation failure |
Check that the system has available memory. |
|
AICPU_DVPP_KERNEL_STATE_SEND_NOTIFY_FAILED = 12 |
Internal notification push failure |
- |
|
AICPU_DVPP_KERNEL_STATE_VPC_OPERATE_FAILED = 13 |
Internal API operation failure |
- |
|
AICPU_DVPP_KERNEL_STATE_CHANNEL_ABNORMAL = 14 |
Abnormal channel |
- |
|
ERR_INVALID_STATE = 0x10001 |
Abnormal VDEC decoder state |
- |
|
ERR_HARDWARE = 0x10002 |
Hardware error, including decoder starts, execution, and stops |
- |
|
ERR_SCD_CUT_FAIL = 0x10003 |
An error occurs when the video stream is divided into multi-frame images. |
Check that the input video stream is valid. |
|
ERR_VDM_DECODE_FAIL = 0x10004 |
Single-frame decoding failure |
Check that the input video stream is valid. |
|
ERR_ALLOC_MEM_FAIL = 0x10005 |
Internal memory allocation failure |
Check whether the system has available memory. If this error is ignored, no decoding result may be output even with stream input. |
|
ERR_ALLOC_DYNAMIC_MEM_FAIL = 0x10006 |
Out-of-range input video resolution and dynamic memory allocation failure |
Check the resolution of the input video stream and whether the system has available memory. If this error is ignored, no decoding result may be output with the continuously sent streams. |
|
ERR_ALLOC_IN_OR_OUT_PORT_MEM_FAIL = 0x10007 |
Internal VDEC input and output buffer allocation failure |
Check whether the system has available memory. If this error is ignored, no decoding result may be output even with stream input. |
|
ERR_BITSTREAM = 0x10008 |
Invalid stream (for example, syntax error, eos retried, or eos detected in the first frame) |
Check that the input video stream is valid. |
|
ERR_VIDEO_FORMAT = 0x10009 |
Incorrect input video format |
Check that the input video format is H.264 or H.265. |
|
ERR_IMAGE_FORMAT = 0x1000a |
Incorrect output format |
Check that the output image format is NV12 or NV21. |
|
ERR_CALLBACK = 0x1000b |
Empty callback function |
Check whether the callback function is empty. |
|
ERR_INPUT_BUFFER = 0x1000c |
Empty input buffer |
Check whether the input buffer is empty. |
|
ERR_INBUF_SIZE = 0x1000d |
Input buffer size ≤ 0 |
Check whether the buffer size is less than or equal to 0. |
|
ERR_THREAD_CREATE_FBD_FAIL = 0x1000e |
Abnormal thread (thread where the callback function returns the decoding result) |
Check that the resources (such as threads and memory) in the system are available. |
|
ERR_CREATE_INSTANCE_FAIL = 0x1000f |
Decoding instance creation failure |
- |
|
ERR_INIT_DECODER_FAIL = 0x10010 |
Decoder initialization failure. For example, the number of decoding instances exceeds the maximum of 16. |
- |
|
ERR_GET_CHANNEL_HANDLE_FAIL = 0x10011 |
Channel handle obtaining failure |
- |
|
ERR_COMPONENT_SET_FAIL = 0x10012 |
Decoding instance setting failure |
Check that the input arguments are correct, for example, the input video format video_format and output frame format image_format. |
|
ERR_COMPARE_NAME_FAIL = 0x10013 |
Decoding instance naming failure |
Check that the input arguments are correct, for example, the input video format video_format and output frame format image_format. |
|
ERR_OTHER = 0x10014 |
Other error |
- |
|
ERR_DECODE_NOPIC = 0x20000 |
Can be ignored. In the interlaced scenario where two fields are transmitted every frame, one of every two transmitted fields has no decoding output. This is expected. The decoding output of an interlaced stream is stored in the output buffers of the odd fields. |
- |
|
0x20001 |
Reference frame count setting failure |
Check that the actual number of reference frames of the stream is the same as the configured number of reference frames. |
|
0x20002 |
Size of VDEC frame buffer setting failure |
Check that the actual width and height of the input stream are the same as the configured width and height. |
|
0xA0058001 |
Invalid device ID |
Check the device ID. |
|
0xA0058002 |
Invalid channel ID |
Check that the input channel ID is correct or whether the total number of channels reaches the upper limit. |
|
0xA0058003 |
Invalid argument, such as an invalid enum value |
Check the incorrect argument based on the logs. |
|
0xA0058004 |
Existing resource |
Check whether the channel has been repeatedly created. |
|
0xA0058005 |
Non-existent channel resource |
Check whether a non-existent channel ID or channel handle is used. |
|
0xA0058006 |
Null pointer |
Check the input of the API based on the logs. |
|
0xA0058007 |
No configuration made to corresponding parameters before the system, device, or channel is enabled. |
Check the input of the API based on the logs. |
|
0xA0058008 |
Unsupported argument or function. |
Check the input of the API based on the logs. |
|
0xA0058009 |
Unallowed operation, for example, an attempt to update a static attribute. |
- |
|
0xA005800C |
Memory allocation failure due to memory insufficiency or other reasons. |
Check whether the system has available memory. If this error is ignored, no decoding result may be output even with stream input. |
|
0xA005800D |
Buffer allocation failure due to too large requested picture buffer or other reasons |
- |
|
0xA005800E |
Empty buffer |
Decoding is not complete, so there is no decoding result in the buffer. Try again after the decoding result is loaded to the buffer. |
|
0xA005800F |
Full buffer |
Stream feeding is too fast, which causes full input buffer. Slow down stream feeding or increase the buffer allocation when creating a channel. |
|
0xA0058010 |
Uninitialized system or unloaded dependent modules |
Check that the system initialization API is called. |
|
0xA0058011 |
Bad address |
- |
|
0xA0058012 |
Busy system |
Check whether the total number of VDEC channels reaches the upper limit. |
|
0xA0058013 |
Not enough buffer |
- |
|
0xA0058014 |
Hardware or software processing timeout |
- |
|
0xA0058015 |
System error |
- |
|
0xA005803F |
Maximum return. The error code value of the module must be less than this value. |
- |
