VENC

API Call Sequence

If video encoding is involved during app development, ensure that your app contains the code logic for such video encoding. For details about the API call sequence, see API Call Sequence.

Figure 1 API call sequence

Optimization of Video Encoding Quality

When implementing the VENC function, you can set basic parameters during channel creation or set advanced parameters by calling the corresponding set API to optimize the video encoding quality. The following optimization settings are available and can be used jointly:

• Settings for H.264 monitoring with the resolution of 720p, GOP of 60, frame rate of 30 FPS, and bit rate of 1 Mbit/s: CBR mode, HI_VENC_SCENE_0, stats_time = 2, profile = 2, and macroblock-level bit rate control = disabled.
• Settings for H.265 movies with the resolution of 1080p, GOP of 30, frame rate of 25 FPS, and bit rate of 2 Mbit/s: CBR mode, HI_VENC_SCENE_1, stats_time = 1, and macroblock-level bit rate control = disabled.

You can optimize the video encoding quality using the following methods:

• Setting basic parameters for overall quality optimization

  The encoding quality of videos with different resolutions is subject to the video frame rate, GOP, and bit rate. When a channel is created by calling acl.himpi.venc_create_chn, you can set the encoding profile and parameters such as the frame rate, GOP, and bit rate in CBR/VBR/AVBR/CVBR/QVBR mode for H.264/H.265 encoding to optimize the video encoding quality.

  • Encoding profile: specified by the profile parameter in the hi_venc_chn_attr["venc_attr"] dictionary.
  • Frame rate: specified by the src_frame_rate input parameter and dst_frame_rate output parameter in the hi_venc_chn_attr["rc_attr"] dictionary.
  • GOP: specified by the gop parameter in the hi_venc_chn_attr["rc_attr"] dictionary.
  • Bit rate: specified by the bit_rate, max_bit_rate, or target_bit_rate parameter in the hi_venc_chn_attr["rc_attr"] dictionary.

  Table 1 Values of the frame rate, GOP, and bit rate in typical scenarios

  Picture Quality/Resolution	Frame Rate (FPS)	GOP	Bit Rate (Mbit/s)
  4K 3840 x 2160/4096 x 2160	25 or 30	It is recommended that the GOP value be an integral multiple of the frame rate. For example, if the frame rate is 25, the recommended GOP value is 25 or 50.	Video data obtaining 8 to 12 for H.264/H.265 streams Show/Livestreaming/Short video 6 to 12 for H.265 streams N/A for H.264 streams Game video 10 to 16 for H.264/H.265 streams
  2K 2560 x 1440	25 or 30	It is recommended that the GOP value be an integral multiple of the frame rate. For example, if the frame rate is 25, the recommended GOP value is 25 or 50.	Video data obtaining 6 to 10 for H.264/H.265 streams Show/Livestreaming/Short video 4.8 to 8 for H.265 streams N/A for H.264 streams Game video 6 to 10 for H.264/H.265 streams
  1080p (Blu-ray) 1920 x 1080	25 or 30	It is recommended that the GOP value be an integral multiple of the frame rate. For example, if the frame rate is 25, the recommended GOP value is 25 or 50.	Video data obtaining 1 to 4 for H.265 streams 2 to 6 for H.264 streams Show/Livestreaming/Short video 1.4 to 3.6 for H.265 streams 2 to 4.8 for H.264 streams Game video 3 to 6 for H.264/H.265 streams
  720p (HD) 1280 x 720	25 or 30	It is recommended that the GOP value be an integral multiple of the frame rate. For example, if the frame rate is 25, the recommended GOP value is 25 or 50.	Video data obtaining 0.8 to 2 for H.265 streams 1 to 3 for H.264 streams Show/Livestreaming/Short video 1 to 2 for H.265 streams 1 to 3 for H.264 streams Game video 2 to 4 for H.264/H.265 streams
  480p/D1_N (SD) 854 x 480/720 x 480	25 or 30	It is recommended that the GOP value be an integral multiple of the frame rate. For example, if the frame rate is 25, the recommended GOP value is 25 or 50.	Video data obtaining 0.3 to 0.7 for H.265 streams 0.6 to 1.4 for H.264 streams Show/Livestreaming/Short video 0.25 to 0.6 for H.265 streams 0.3 to 0.7 for H.264 streams Game video N/A
  576p/D1 (SD) 720 x 576	25 or 30	It is recommended that the GOP value be an integral multiple of the frame rate. For example, if the frame rate is 25, the recommended GOP value is 25 or 50.	Video data obtaining 0.3 to 0.7 for H.265 streams 0.6 to 1.4 for H.264 streams Show/Livestreaming/Short video 0.25 to 0.6 for H.265 streams 0.3 to 0.7 for H.264 streams Game video N/A
  270p (smooth) 480 x 270	25 or 30	It is recommended that the GOP value be an integral multiple of the frame rate. For example, if the frame rate is 25, the recommended GOP value is 25 or 50.	Video data obtaining N/A Show/Livestreaming/Short video 0.2 for H.265 streams 0.3 for H.264 streams Game video N/A
  CIF P/N 352 x 288/320 x 240	25 or 30	It is recommended that the GOP value be an integral multiple of the frame rate. For example, if the frame rate is 25, the recommended GOP value is 25 or 50.	Video data obtaining 0.25 for H.264/H.265 streams Show/Livestreaming/Short video N/A Game video N/A

• Setting advanced parameters for quality tuning

  You can call the APIs listed in the following table to set the bit rate control mode, macroblock-level bit rate control parameters, and encoding scene mode to tune video encoding quality.

  Table 2 Advanced settings

  Item	API	Parameter	Description
  Bit rate control mode	acl.himpi.venc_create_chn	rc_mode in the hi_venc_chn_attr["rc_attr"] dictionary	Set this parameter to CBR if a stable bit rate or a large peak signal to noise ratio (PSNR) and qualified bit rate is required. Set this parameter to VBR to save the bit rate and improve the subjective quality of encoding. Set this parameter to AVBR to save the bit rate and improve the subjective quality of encoding with lots of still images in the scene. Set this parameter to QVBR if a large PSNR is required and there is no strict requirement on the bit rate increase. Set this parameter to CVBR to save the bit rate and improve the subjective quality of encoding when the bandwidth and storage space can also be used for optimization.
  Statistical time of the bit rate control model	acl.himpi.venc_create_chn	stats_time in the hi_venc_chn_attr["rc_attr"] dictionary	Set this parameter to a large value in the scenario where long-term bit rate stability is required without considering short-term fluctuations, for example, DVR storage. This setting increases the threshold for re-encoding to cut the number of re-encoding times, but also increases bit rate fluctuations at the same time.
  Macroblock-level bit rate control	acl.himpi.venc_set_rc_param	threshold_i, threshold_p, threshold_b, direction, and row_qp_delta in the hi_venc_rc_param dictionary.	Disable this function when encoding complex images with rich details, or when objective indicators such as the PSNR are concerned.
  Start QP value of the first frame	acl.himpi.venc_create_chn	first_frame_start_qp in the hi_venc_rc_param dictionary	In typical scenarios, if the configured bit rate is less than the reference value provided in Table 1 and the first frame of the encoded video is blurry, you are advised to set first_frame_start_qp to the mean value of [min_i_qp, max_i_qp]. For example, if [min_i_qp, max_i_qp] is [30, 40], set first_frame_start_qp to 35. In addition, set max_reencode_times to 0.
  Encoding scene mode	acl.himpi.venc_set_scene_mode	hi_venc_scene_mode class	Set this parameter to HI_VENC_SCENE_0 for security scenes. Set this parameter to HI_VENC_SCENE_1 for autonomous driving, livestreaming, gaming, animations, and movies.

Sample Code

After APIs are called, add an exception handling branch, and record error logs and warning logs. The following is a code snippet of key steps only, which is not ready to use.

  1
  2
  3
  4
  5
  6
  7
  8
  9
 10
 11
 12
 13
 14
 15
 16
 17
 18
 19
 20
 21
 22
 23
 24
 25
 26
 27
 28
 29
 30
 31
 32
 33
 34
 35
 36
 37
 38
 39
 40
 41
 42
 43
 44
 45
 46
 47
 48
 49
 50
 51
 52
 53
 54
 55
 56
 57
 58
 59
 60
 61
 62
 63
 64
 65
 66
 67
 68
 69
 70
 71
 72
 73
 74
 75
 76
 77
 78
 79
 80
 81
 82
 83
 84
 85
 86
 87
 88
 89
 90
 91
 92
 93
 94
 95
 96
 97
 98
 99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130

# 1. Obtain the run mode of the software stack. Different run modes lead to different API call sequences (for example, whether data transfer is required).
run_mode, ret = acl.rt.get_run_mode()

# 2. Perform initialization.
ret = acl.init()

# 3. Allocate runtime resources.

# 4. Initialize the media data processing system.
ret = acl.himpi.sys_init()

# 5. Set VENC parameters.
param = {'mod_type':HI_VENC_MOD_H265}
param, ret = acl.himpi.venc_get_mod_param(param)
param['jpeg_mod_param']['one_stream_buf'] = 1
ret = acl.himpi.venc_set_mod_param(param)

# 6. Create a channel.
channel_id = 0
venc_attr = {'type': HI_VENC_MOD_H265, 'profile': 0,
             'max_pic_width': 128, 'pic_width': 128,
             'max_pic_height': 128, 'pic_height': 128,
             'buf_size': 1024 * 1024 * 2, 'is_by_frame': 1}
rc_attr = {'rc_mode':HI_VENC_RC_MODE_H265_VBR,
           'h265_vbr':{'gop': 30, 'stats_time': 1,
                       'src_frame_rate': 30, 'dst_frame_rate': 30,
                       'max_bit_rate': 4000}}
gop_attr = {'gop_mode':0, 'normal_p':{'ip_qp_delta':3}}
attr = {'venc_attr':venc_attr, 'rc_attr':rc_attr, 'gop_attr':gop_attr}
ret = acl.himpi.venc_create_chn(channel_id, attr)

# 7. Notify the encoder to start receiving input data.
recv_param = {'recv_pic_num':-1}
ret = acl.himpi.venc_start_chn(channel_id, recv_param)

# 8. Send the input data.
# 8.1 Allocate the input buffer.
input_size = 128 * 128 * 3 // 2
input_addr, ret = acl.himpi.dvpp_malloc(0, input_size);

# If the run mode is ACL_HOST, allocate the host buffer, load the input data into the host buffer, and call acl.rt.memcpy to transfer the host data to the device. After the data transfer is complete, free the host buffer in a timely manner. In other modes, directly load the input data into the device buffer.
# Directly load the input data into the device buffer.
if run_mode == ACL_HOST:
    # Read the input image to the buffer.
    jpege_file = np.fromfile(jpege_file_path, dtype=np.byte)
    jpege_file_size = jpege_file.itemsize * jpege_file.size
    
    bytes_data = jpege_file.tobytes()
    jpege_file_ptr = acl.util.bytes_to_ptr(bytes_data)
    # Transfer data.
    ret = acl.rt.memcpy(input_addr, input_size, jpege_file_ptr, jpege_file_size, ACL_MEMCPY_HOST_TO_DEVICE)
else:
    # Read the input image to the buffer.
    jpege_file = np.fromfile(jpege_file_path, dtype=np.byte)
    jpege_file_size = jpege_file.itemsize * jpege_file.size
    
    bytes_data = jpege_file.tobytes()
    jpege_file_ptr = acl.util.bytes_to_ptr(bytes_data)
    # Transfer data.
    ret = acl.rt.memcpy(input_addr, input_size, jpege_file_ptr, jpege_file_size, ACL_MEMCPY_DEVICE_TO_DEVICE)

# 8.2 Send the input data and start encoding.
v_frame = {'width': 128,
           'height': 128,
           'field': HI_VIDEO_FIELD_FRAME,
           'pixel_format': HI_PIXEL_FORMAT_YUV_SEMIPLANAR_420,
           'video_format': HI_VIDEO_FORMAT_LINEAR,
           'compress_mode': HI_COMPRESS_MODE_NONE,
           'dynamic_range': HI_DYNAMIC_RANGE_SDR8,
           'color_gamut': HI_COLOR_GAMUT_BT709,
           'header_stride': [0, 0, 0],
           'width_stride': [128, 0, 0],
           'height_stride': [0, 0, 0],
           'header_phys_addr': [0, 0, 0],
           'phys_addr': [0, 0, 0],
           'header_virt_addr': [0, 0, 0],
           'virt_addr': [input_addr, 0, 0],
           'time_ref': 0,'pts': cur_time}
frame = {'v_frame':v_frame, 'pool_id':0, 'mod_id':HI_ID_VENC}
ret = acl.himpi.venc_send_frame(channel_id, frame, 0)

# 9. Obtain the encoding result.
# 9.1 Handle the encoding completion event through epoll.
fd = acl.himpi.venc_get_fd(channel_id)
epoll_fd, ret = acl.himpi.sys_create_epoll(10)

event['data'] = fd
event['events'] = HI_DVPP_EPOLL_IN
ret = acl.himpi.sys_ctl_epoll(epoll_fd, HI_DVPP_EPOLL_CTL_ADD, fd, event)

# Before the encoding is complete, timeout occurs. The next step is not performed until the encoding is complete.
events, eventCount, ret = acl.himpi.sys_wait_epoll(epoll_fd, 3, 1000);

# 9.2 Obtain the encoding result.
status, ret = acl.himpi.venc_query_status(channel_id)
stream = {'pack_cnt': status['cur_packs']}
stream, ret = acl.himpi.venc_get_stream(self.channel_id, stream, 1000)
# 9.3 If the run mode is ACL_HOST and the host needs to use the encoded streams, allocate the host buffer and call acl.rt.memcpy to transfer the output streams from the device to the host.
# Obtain the encoded streams.
if run_mode == ACL_HOST:
    # Allocate the host buffer.
    output_buffer, ret= acl.rt.malloc_host(output_size)
    # Transfer data.
    ret = acl.rt.memcpy(output_buffer, output_size, stream['pack'][0]['addr'], output_size, ACL_MEMCPY_DEVICE_TO_HOST)
    # ......
    # After data is used, free the buffer in a timely manner.
    ret = acl.rt.free_host(output_buffer)
else:
    # You can directly use the encoded streams, which are stored in the buffer specified by stream['pack'][0]['addr'].
    # ......

# 10. Free the input buffer and release the output streams.
ret = acl.himpi.dvpp_free(input_addr)
ret = acl.himpi.venc_release_stream(channel_id, stream)

# 11. Notify the encoder to stop receiving input data.
ret = acl.himpi.venc_stop_chn(channel_id)
ret = acl.himpi.sys_ctl_epoll(epoll_fd, HI_DVPP_EPOLL_CTL_DEL, fd, event)
ret = acl.himpi.sys_close_epoll(epoll_fd)

# 12. Destroy the channel.
ret = acl.himpi.venc_destroy_chn(channel_id)

# 13. Deinitialize the media data processing system.
ret = acl.himpi.sys_exit()

# 14. Deallocate runtime resources.

# 15. Perform deinitialization.
ret = acl.finalize()