文档

Typical Functions of VPC

This section takes image cropping and resizing as an example to explain the API call sequence when the VPC processes images and provides sample code of typical functions to help you better understand the API call sequence.

Vision Preprocessing Core (VPC) supports image cropping, resizing, pasting, and format conversion. For details, see Functions. For details about the input and output restrictions of the VPC function, see Restrictions.

Typical API Call Sequence (Image Resizing)

If image cropping and resizing are involved during app development, ensure that your app contains the code logic for such image processing. For details about the API call sequence, see API Call Sequence.

If the DVPP APIs are called on the host, the image processing result will be stored in the device memory. The resultant data needs to be transferred back to the host so that it can be accessed.

Figure 1 Typical API call sequence

The current system supports image cropping and resizing. The key APIs are described as follows:

  1. Call hi_mpi_sys_init to initialize the media data processing system.
  2. Call hi_mpi_vpc_create_chn to create a channel.
  3. Call hi_mpi_dvpp_malloc to allocate the device buffer to store the input or output data.

    On the Atlas 200I/500 A2 inference products , aclrtMalloc can be used to allocate buffers.

    On the Atlas A2 training products / Atlas A2 inference products , aclrtMalloc can be used to allocate buffers.

    hi_mpi_dvpp_malloc allocates a dedicated buffer for processing media data. However, the address space of the dedicated buffer is limited. If buffer planning is cared or buffer resources are limited, you are advised to allocate buffer by calling aclrtMalloc.

  4. Perform image cropping and resizing. Resizing is used as an example. For details about other functions (such as format conversion and pyramid), see the API description in VPC.

    Call hi_mpi_vpc_resize to resize the image. hi_mpi_vpc_resize is an asynchronous API. The API call only delivers a task. You also need to call hi_mpi_vpc_get_process_result to obtain the processing result.

    • You can call hi_mpi_vpc_get_process_result in the same thread as hi_mpi_vpc_resize, or start a new thread. Multiple threads can improve efficiency, but require inter-thread synchronization.
    • To reformat the input image during cropping and resizing, you can set different formats for the input and output images.
    • Call aclrtGetRunMode to obtain the run mode of the software stack. If the run mode is ACL_HOST and the image data output by VPC needs to be displayed on the host, allocate the host buffer and call aclrtMemcpy to transfer the output image data from the device to the host. If the data does not need to be displayed on the host, it can be used as the input of model inference. For details about model inference, see Model Inference with Static-Shape Inputs and Dynamic AIPP Model Inference.
  5. Call hi_mpi_dvpp_free to free the input and output buffers.

    On the Atlas 200I/500 A2 inference products , if aclrtMalloc is used to allocate buffers, aclrtFree needs to be used to free buffers.

    On the Atlas A2 training products / Atlas A2 inference products , if aclrtMalloc is used to allocate buffers, aclrtFree needs to be used to free buffers.

  6. Call hi_mpi_vpc_destroy_chn to destroy the channel.
  7. Call hi_mpi_sys_exit to deinitialize the media data processing system.

Sample Code for Image Resizing

You can view the complete code in Media Data Processing V2 (VPC Cropping, Pasting, and Resizing).

This section focuses on the code logic of image resizing. For details about how to perform initialization and deinitialization, see Initialization and Deinitialization. For details about how to allocate and deallocate runtime resources, see Runtime Resource Allocation and Deallocation.

After APIs are called, you need to add exception handling branches and record error logs and info logs. The following is a code snippet of key steps only, which is not ready to be built or run.

  1
  2
  3
  4
  5
  6
  7
  8
  9
 10
 11
 12
 13
 14
 15
 16
 17
 18
 19
 20
 21
 22
 23
 24
 25
 26
 27
 28
 29
 30
 31
 32
 33
 34
 35
 36
 37
 38
 39
 40
 41
 42
 43
 44
 45
 46
 47
 48
 49
 50
 51
 52
 53
 54
 55
 56
 57
 58
 59
 60
 61
 62
 63
 64
 65
 66
 67
 68
 69
 70
 71
 72
 73
 74
 75
 76
 77
 78
 79
 80
 81
 82
 83
 84
 85
 86
 87
 88
 89
 90
 91
 92
 93
 94
 95
 96
 97
 98
 99
100
101
 
// 1. Perform initialization.

// 2. Allocate runtime resources.

// 3. Initialize the media data processing system.
int32_t ret = hi_mpi_sys_init();

// 4. Create a channel.
hi_vpc_chn chnId;
hi_vpc_chn_attr stChnAttr;
ret = hi_mpi_vpc_sys_create_chn(&chnId, &stChnAttr);

// 5. Perform resizing.
// 5.1 Construct a struct for storing input image information.
hi_vpc_pic_info inputPic;
inputPic.picture_width = 1920;
inputPic.picture_height = 1080;
inputPic.picture_format = HI_PIXEL_FORMAT_YUV_SEMIPLANAR_420;
inputPic.picture_width_stride = 1920;
inputPic.picture_height_stride = 1080;
inputPic.picture_buffer_size = inputPic.picture_width_stride * inputPic.picture_height_stride * 3 / 2;

// 5.2 Prepare input image data.
// Allocate the device buffer for media data processing.
ret = hi_mpi_dvpp_malloc(0, &inputPic.picture_address, inputPic.picture_buffer_size);

// If the run mode is ACL_HOST, allocate the host buffer, load the input image data into the host buffer, and call aclrtMemcpy to transfer the host image data to the device. After the data transfer is complete, free the host buffer in a timely manner. In other modes, directly load the input image data into the device buffer.
// runMode indicates the running mode of the software stack, which can be obtained through aclrtGetRunMode.
if (runMode == ACL_HOST) {
    void* inputAddr = nullptr;
    // Allocate the host buffer.
    aclRet = aclrtMallocHost(&inputAddr, inputPic.picture_buffer_size);
    // Load the input image into the buffer. The ReadPicFile function is defined by the user.
    ReadPicFile(picName, inputAddr, inputPic.picture_buffer_size);
    // Transfer data.
    aclRet = aclrtMemcpy(inputPic.picture_address, inputPic.picture_buffer_size, inputAddr,
                         inputPic.picture_buffer_size, ACL_MEMCPY_HOST_TO_DEVICE);
    // Free the buffer in a timely manner after data transfer is complete.
    aclrtFreeHost(inputAddr);
    inputAddr = nullptr;
} 
else {
    // Load the input image into the buffer. The ReadPicFile function is defined by the user.
    ReadPicFile(picName, inputPic.picture_address, inputPic.picture_buffer_size);
    }

// 5.3 Construct a struct for storing output image information.
hi_vpc_pic_info outputPic;
outputPic.picture_width = 960;
outputPic.picture_height  = 540;
outputPic.picture_format = HI_PIXEL_FORMAT_YUV_SEMIPLANAR_420;
outputPic.picture_width_stride = 960;
outputPic.picture_height_stride = 540;
outputPic.picture_buffer_size = outputPic.picture_width_stride * outputPic.picture_height_stride * 3 / 2;
ret = hi_mpi_dvpp_malloc(0, &outputPic.picture_address, outputPic.picture_buffer_size);

// Initialize the buffer.
if (runMode == ACL_HOST) {
    aclRet = aclrtMemset(outputPic.picture_address, outputPic.picture_buffer_size, 0, outputPic.picture_buffer_size);
} else {
     memset(outputPic.picture_address, 0, outputPic.picture_buffer_size);
}

// 5.4 Call the image resizing API.
uint32_t taskID = 0;
ret = hi_mpi_vpc_resize(chnId, &inputPic, &outputPic, 0, 0, 0, &taskID, -1);

// 5.5 Wait until the task processing is complete. The output image data is stored in the buffer specified by outputPic.picture_address.
uint32_t taskIDResult = taskID;
ret = hi_mpi_vpc_get_process_result(chnId, taskIDResult, -1);

// 5.6 If the run mode is ACL_HOST and the image data output by VPC needs to be displayed on the host, allocate the host buffer and call aclrtMemcpy to transfer the output image data from the device to the host. If the data does not need to be displayed on the host, it can be used as the input of model inference.
if (g_runMode == ACL_HOST) {
    hi_vpc_pic_info outputPicHost = outputPic;
    aclRet = aclrtMallocHost(&outputPicHost.picture_address, outputPic.picture_buffer_size);
    aclRet = aclrtMemcpy(outputPicHost.picture_address, outputPic.picture_buffer_size, outputPic.picture_address,outputPic.picture_buffer_size, ACL_MEMCPY_DEVICE_TO_HOST);
    // ......
    // Free the host buffer after the data on the host is used.
    aclrtFreeHost(outputPicHost.picture_address);
    outputPicHost.picture_address = nullptr;
} else {
    // You can directly use the output image data of VPC in the buffer specified by outputPic.picture_address.
    // TODO: inference-related code logic
}

// 5.7 Free the input and output buffers.
ret = hi_mpi_dvpp_free(inputPic.picture_address);
ret = hi_mpi_dvpp_free(outputPic.picture_address);

// 6. Destroy the channel.
ret = hi_mpi_vpc_destroy_chn(chnId);

// 7. Deinitialize the media data processing system.
ret = hi_mpi_sys_exit();

// 8. Deallocate runtime resources.

// 9. Perform deinitialization.


// ....

Sample Code for Image Cropping

You can view the complete code in Media Data Processing V2 (VPC Cropping, Pasting, and Resizing).

This section focuses on the code logic of image resizing. For details about how to perform initialization and deinitialization, see Initialization and Deinitialization. For details about how to allocate and deallocate runtime resources, see Runtime Resource Allocation and Deallocation.

After APIs are called, you need to add exception handling branches and record error logs and info logs. The following is a code snippet of key steps only, which is not ready to be built or run.

  1
  2
  3
  4
  5
  6
  7
  8
  9
 10
 11
 12
 13
 14
 15
 16
 17
 18
 19
 20
 21
 22
 23
 24
 25
 26
 27
 28
 29
 30
 31
 32
 33
 34
 35
 36
 37
 38
 39
 40
 41
 42
 43
 44
 45
 46
 47
 48
 49
 50
 51
 52
 53
 54
 55
 56
 57
 58
 59
 60
 61
 62
 63
 64
 65
 66
 67
 68
 69
 70
 71
 72
 73
 74
 75
 76
 77
 78
 79
 80
 81
 82
 83
 84
 85
 86
 87
 88
 89
 90
 91
 92
 93
 94
 95
 96
 97
 98
 99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
 
// 1. Perform initialization.

// 2. Allocate runtime resources.

// 3. Initialize the media data processing system.
int32_t ret = hi_mpi_sys_init();

// 4. Create a channel.
hi_vpc_chn chnId;
hi_vpc_chn_attr stChnAttr;
ret = hi_mpi_vpc_sys_create_chn(&chnId, &stChnAttr);

// 5. Perform image cropping.
// 5.1 Construct a struct for storing input image information.
hi_vpc_pic_info inputPic;
inputPic.picture_width = 1920;
inputPic.picture_height = 1080;
inputPic.picture_format = HI_PIXEL_FORMAT_YUV_SEMIPLANAR_420;
inputPic.picture_width_stride = 1920;
inputPic.picture_height_stride = 1080;
inputPic.picture_buffer_size = inputPic.picture_width_stride * inputPic.picture_height_stride * 3 / 2;

// 5.2 Prepare input image data.
// Allocate the device buffer for media data processing.
ret = hi_mpi_dvpp_malloc(0, &inputPic.picture_address, inputPic.picture_buffer_size);

// If the run mode is ACL_HOST, allocate the host buffer, load the input image data into the host buffer, and call aclrtMemcpy to transfer the host image data to the device. After the data transfer is complete, free the host buffer in a timely manner. In other modes, directly load the input image data into the device.
// runMode indicates the running mode of the software stack, which can be obtained through aclrtGetRunMode.
if (runMode == ACL_HOST) {
    void* inputAddr = nullptr;
    // Allocate the host buffer.
    aclRet = aclrtMallocHost(&inputAddr, inputPic.picture_buffer_size);
    // Load the input image into the buffer. The ReadPicFile function is defined by the user.
    ReadPicFile(picName, inputAddr, inputPic.picture_buffer_size);
    // Transfer data.
    aclRet = aclrtMemcpy(inputPic.picture_address, inputPic.picture_buffer_size, inputAddr,
                          inputPic.picture_buffer_size, ACL_MEMCPY_HOST_TO_DEVICE);
    // Free the buffer in a timely manner after data transfer is complete.
    aclrtFreeHost(inputAddr);
    inputAddr = nullptr;
} else {
    // Load the input image into the buffer. The ReadPicFile function is defined by the user.
    ReadPicFile(picName, inputPic.picture_address, inputPic.picture_buffer_size);
}

// 5.3 Construct a struct for storing output image information.
// This parameter indicates the number of crop ROIs.
uint32_t multiCount = 1;
// The size of the cropRegionInfos array must be consistent with the number of crop ROIs.
hi_vpc_crop_region_info cropRegionInfos[1];
hi_vpc_pic_info outputPic;
for (uint32_t i = 0; i < multiCount; i++) {
    outputPic.picture_width = 960;
    outputPic.picture_height  = 540;
    outputPic.picture_format = HI_PIXEL_FORMAT_YUV_SEMIPLANAR_420;
    outputPic.picture_width_stride = 960;
    outputPic.picture_height_stride = 540;
    outputPic.picture_buffer_size = outputPic.picture_width_stride * outputPic.picture_height_stride * 3 / 2;
    ret = hi_mpi_dvpp_malloc(0, &outputPic.picture_address, outputPic.picture_buffer_size);

// Initialize the buffer.
    if (runMode == ACL_HOST) {
        aclRet = aclrtMemset(outputPic.picture_address, outputPic.picture_buffer_size, 0, outputPic.picture_buffer_size);
    } else {
        memset(outputPic.picture_address, 0, outputPic.picture_buffer_size);
    }
    
    // Crop a 960 x 540 ROI from the input image with the upper-left corner as the origin.
    cropRegionInfos[i].dest_pic_info = outputPic;
    cropRegionInfos[i].crop_region.left_offset = 0;
    cropRegionInfos[i].crop_region.top_offset = 0;
    cropRegionInfos[i].crop_region.crop_width = 960;
    cropRegionInfos[i].crop_region.crop_height = 540;
}

// 5.4 Call the image cropping API.
uint32_t taskID = 0;
ret = hi_mpi_vpc_crop(chnId, &inputPic, cropRegionInfos, 1, &taskID, -1);

// 5.5 Wait until the task processing is complete. The output image data is stored in the buffer specified by outputPic.picture_address.
uint32_t taskIDResult = taskID;
ret = hi_mpi_vpc_get_process_result(chnId, taskIDResult, -1);

// 5.6 If the run mode is ACL_HOST and the image data output by VPC needs to be displayed on the host, allocate the host buffer and call aclrtMemcpy to transfer the output image data from the device to the host. If the data does not need to be displayed on the host, it can be used as the input of model inference.
if (g_runMode == ACL_HOST) {
    hi_vpc_pic_info outputPicHost = outputPic;
    aclRet = aclrtMallocHost(&outputPicHost.picture_address, outputPic.picture_buffer_size);
    aclRet = aclrtMemcpy(outputPicHost.picture_address, outputPic.picture_buffer_size, outputPic.picture_address,outputPic.picture_buffer_size, ACL_MEMCPY_DEVICE_TO_HOST);
    // ......
    // Free the host buffer after the data on the host is used.
    aclrtFreeHost(outputPicHost.picture_address);
    outputPicHost.picture_address = nullptr;
} else {
    // You can directly use the output image data of VPC in the buffer specified by outputPic.picture_address.
    // TODO: inference-related code logic

}

// 5.7 Free the input and output buffers.
ret = hi_mpi_dvpp_free(inputPic.picture_address);
inputPic.picture_address = nullptr;
for (uint32_t i = 0; i < multiCount; i++) {
     hi_mpi_dvpp_free(cropRegionInfos[i].dest_pic_info.picture_address);
     cropRegionInfos[i].dest_pic_info.picture_address = nullptr;
}

// 6. Destroy the channel.
ret = hi_mpi_vpc_destroy_chn(chnId);

// 7. Deinitialize the media data processing system.
ret = hi_mpi_sys_exit();

// 8. Deallocate runtime resources.

// 9. Perform deinitialization.

// ....
我们使用cookie来确保您的高速浏览体验。继续浏览本站，即表示您同意我们使用cookie。查看详情