Image Classification with ResNet-50 (Image Decoding+Cropping and Resizing+Image Encoding+Synchronous Inference)

Sample Obtaining

Refer to vpc_jpeg_resnet50_imagenet_classification to obtain the sample.

Description

This sample shows how to classify images based on the Caffe ResNet-50 network (single input with batch size = 1).

According to the arguments of the app, the following functions can be implemented:

Encodes a YUV420SP image into the .jpg format.
Decodes two .jpg images into YUV420SP (NV12) images, resizes the images, performs model inference to obtain the inference results of the two images, processes the inference results, and outputs the class indexes with the top confidence values and the sum of the top 5 confidence values.
Decodes two .jpg images into YUV420SP (NV12) images, crops selected ROIs from the images, performs model inference to obtain the inference results of the two images, processes the inference results, and outputs the class indexes with the top confidence values and the sum of the top 5 confidence values.
Decodes two .jpg images into YUV420SP (NV12) images, crops selected ROIs from the images and pastes each cropped image in the canvas, performs model inference to obtain the inference results of the two images, processes the inference results, and outputs the class indexes with the top confidence values and the sum of the top 5 confidence values.
Resizes the 8192 x 8192 image in YUV420SP (NV12) format to 4000 x 4000.

In this sample, an .om offline model adapted to the Ascend AI Processor is used for inference. During model conversion, set CSC parameters to convert YUV420SP images into RGB images to meet the input requirements of the model.

Principles

The following table lists the key functions involved in this sample.

Initialization	aclInit: initializes AscendCL. aclFinalize: deinitializes AscendCL.
Device Management	aclrtSetDevice: sets the compute device. aclrtGetRunMode: obtains the run mode of the software stack. The internal processing varies depending on the run mode. aclrtResetDevice: resets the compute device and cleans up all resources associated with the device.
Stream Management	aclrtCreateStream: creates a stream. aclrtDestroyStream: destroys a stream. aclrtSynchronizeStream: waits for stream tasks to complete.
Memory Management	aclrtMallocHost: allocates host memory. aclrtFreeHost: frees host memory. aclrtMalloc: allocates device memory. aclrtFree: frees device memory. In media data processing, if you need to allocate device memory to store the input or output data, call acldvppMalloc to allocate memory and call acldvppFree to free up memory.
Data Transfer	aclrtMemcpy (used when the app runs on the host): Transfers decode source data from the host to the device. Transfers the inference result from the device to the host. Data transfer is not required if your app runs in the board environment.
Media Data Processing V1	Image encoding acldvppJpegEncodeAsync: encodes YUV420SP images into .jpg images. Image decoding acldvppJpegDecodeAsync: decodes .jpg images into YUV420SP images. Resizing acldvppVpcResizeAsync: resizes the YUV420SP input. Cropping acldvppVpcCropResizeAsync: crops a selected ROI from the input image and loads the cropped image to the output buffer. Cropping and pasting acldvppVpcCropResizePasteAsync: crops a selected ROI from the input image according to cropArea and loads the cropped image to the output buffer.
Model Inference	aclmdlLoadFromFileWithMem: loads a model from an .om file. aclmdlExecute: performs model inference. Before inference, use the CSC parameters in the .om file to convert a YUV420SP image into an RGB image. aclmdlUnload: unloads a model.

Directory Structure

The sample directory is organized as follows:

├── caffe_model
│   ├── aipp.cfg        //Configuration file with CSC parameters, used for model conversion

├── data
│   ├── persian_cat_1024_1536_283.jpg            //Test image. Obtain the test image according to the guide and save it to the data directory.
│   ├── wood_rabbit_1024_1061_330.jpg            //Test image. Obtain the test image according to the guide and save it to the data directory.
│   ├── wood_rabbit_1024_1068_nv12.yuv            //Test image. Obtain the test image according to the guide and save it to the data directory.
│   ├── dvpp_vpc_8192x8192_nv12.yuv            //Test image. Obtain the test image according to the guide and save it to the data directory.

├── inc
│   ├── dvpp_process.h               //Header file that declares functions related to media data processing
│   ├── model_process.h              //Header file that declares functions related to model processing
│   ├── sample_process.h              //Header file that declares functions related to resource initialization and destruction
│   ├── utils.h                       //Header file that declares common functions (such as the file reading function)

├── src
│   ├── acl.json         //Configuration file for system initialization
│   ├── CMakeLists.txt         //Build script
│   ├── dvpp_process.cpp       //Implementation file of functions related to media data processing
│   ├── main.cpp               //Implementation file of the main function, for image classification
│   ├── model_process.cpp      //Implementation file of model processing functions
│   ├── sample_process.cpp     //Implementation file of functions related to resource initialization and destruction
│   ├── utils.cpp              //Implementation file of common functions (such as the file reading function)

├── .project     //Project information file, including the project type, project description, and type of the target device
├── CMakeLists.txt    //Build script that calls the CMakeLists file in the src directory

App Build and Run (Ascend EP Mode)

Refer to vpc_jpeg_resnet50_imagenet_classification to obtain the sample. View the README file in the sample.

App Build and Run (Ascend RC Mode)

Refer to vpc_jpeg_resnet50_imagenet_classification to obtain the sample. View the README file in the sample.

Parent topic: App Samples