Image Classification with ResNet-50 (Image Decoding+Resizing+Synchronous Inference)

Sample Obtaining

Refer to vpc_resnet50_imagenet_classification to obtain the sample.

Description

This sample shows how to classify images based on the Caffe ResNet-50 network (single input with batch size = 1).

Convert the model file of the Caffe ResNet-50 network to an .om offline model adapted to the Ascend AI Processor. In the sample, load the .om file, decode, resize, and infer two .jpg images, process the obtained inference results, and output the class index with the top confidence value of each image.

During model conversion, set CSC parameters to convert YUV420SP images into RGB images to meet the input requirements of the model.

Figure 1 Sample diagram

Principles

The following table lists the key functions involved in this sample.

Initialization	aclInit: initializes AscendCL. aclFinalize: deinitializes AscendCL.
Device Management	aclrtSetDevice: sets the compute device. aclrtGetRunMode: obtains the run mode of the software stack. The internal processing varies depending on the run mode. aclrtResetDevice: resets the compute device and cleans up all resources associated with the device.
Stream Management	aclrtCreateStream: creates a stream. aclrtDestroyStream: destroys a stream. aclrtSynchronizeStream: waits for stream tasks to complete.
Memory Management	aclrtMallocHost: allocates host memory. aclrtFreeHost: frees host memory. aclrtMalloc: allocates device memory. aclrtFree: frees device memory. In media data processing, if you need to allocate device memory to store the input or output data, call acldvppMalloc to allocate memory and call acldvppFree to free up memory.
Data Transfer	aclrtMemcpy (used when the app runs on the host): Transfers decode source data from the host to the device. Transfers the inference result from the device to the host. Data transfer is not required if your app runs in the board environment.
Media Data Processing V1	acldvppJpegDecodeAsync: decodes .jpg images into YUV420SP images. acldvppVpcResizeAsync: resizes YUV420SP images to 224 x 224.
Model Inference	aclmdlLoadFromFileWithMem: loads a model from an .om file. aclmdlExecute: performs model inference. Before inference, use the CSC parameters in the .om file to convert a YUV420SP image into an RGB image. aclmdlUnload: unloads a model.
Data Postprocessing (Single-Operator Execution)	Cast: casts the data type of the inference result from float32 to float16; ArgMaxD: searches for the class indexes with the top confidence values in the inference result. aclopSetModelDir: loads the single-operator model file; aclopCast: executes the Cast operator; aclopExecuteV2: executes the ArgMaxD operator.

Directory Structure

The sample directory is organized as follows:

├── caffe_model
│   ├── aipp.cfg        //Configuration file with CSC parameters, used for model conversion

├── data
│   ├── dog1_1024_683.jpg            //Test image. Obtain the test image according to the guide and save it to the data directory.
│   ├── dog2_1024_683.jpg            //Test image. Obtain the test image according to the guide and save it to the data directory.

├── inc
│   ├── dvpp_process.h               //Header file that declares functions related to media data processing
│   ├── model_process.h              //Header file that declares functions related to model processing
│   ├── sample_process.h               //Header file that declares functions related to resource initialization and destruction
│   ├── singleOp_process.h              //Header file that declares the functions related to single-operator execution
│   ├── utils.h                       //Header file that declares common functions (such as the file reading function)

├── out
│   ├── op_models
│   ├──├──op_list.json              //Description information of operators ArgMaxD and Cast

├── src
│   ├── acl.json         //Configuration file for system initialization
│   ├── CMakeLists.txt         //Build script
│   ├── dvpp_process.cpp       //Implementation file of functions related to media data processing
│   ├── main.cpp               //Implementation file of the main function, for image classification
│   ├── model_process.cpp      //Implementation file of model processing functions
│   ├── sample_process.cpp     //Implementation file of functions related to resource initialization and destruction
│   ├── singleOp_process.cpp   //Implementation file of functions related to single-operator execution
│   ├── utils.cpp              //Implementation file of common functions (such as the file reading function)

├── .project     //Project information file, including the project type, project description, and type of the target device
├── CMakeLists.txt    //Build script that calls the CMakeLists file in the src directory

App Build and Run (Ascend EP)

Refer to vpc_resnet50_imagenet_classification to obtain the sample. View the README file in the sample.

App Build and Run (Ascend RC Mode)

Refer to vpc_resnet50_imagenet_classification to obtain the sample. View the README file in the sample.

App Build and Run (Open Ctrl CPU Form)

Convert your model.
1. Log in to the development environment as the running user.
2. Prepare the environment, including obtaining tools and setting environment variables. For details, see ATC Instructions.
3. Prepare data.
  Download the *.prototxt model file and *.caffemodel weight file of the ResNet-50 network and upload the files to /caffe_model under the vpc_resnet50_imagenet_classification sample directory in the development environment as the running user. If the directory does not exist, create it.
  - ResNet-50 network model file (*.prototxt): Click here to download the file.
  - ResNet-50 weight file (*.caffemodel): Click here to download the file.
4. Convert the ResNet-50 network to an *.om offline model adapted to the Ascend AI Processor. Also set CSC parameters to convert YUV420SP images to RGB images.
  Go to the vpc_resnet50_imagenet_classification sample directory and run the following command:
```
atc --model=caffe_model/resnet50.prototxt --weight=caffe_model/resnet50.caffemodel --framework=0 --output=model/resnet50_aipp --soc_version=${soc_version} --insert_op_conf=caffe_model/aipp.cfg
```
  - --soc_version: indicates the version of the .
    Set --soc_version to the version, which is exactly the .ini file name stored in atc/data/platform_config in the ATC installation path.
  - --output: specifies the directory for storing the generated resnet50_aipp.om file, that is, /model under the sample directory. The default path in the command example is recommended. To specify another path, change the value of omModelPath in sample_process.cpp before building the code.
```
const char* omModelPath = "../model/resnet50_aipp.om";
```
  For details about the available options, see ATC Instructions.
5. Build the operator description information (*.json files) of the Cast and ArgMaxD operators into an offline model adapted to the Ascend AI Processor (*.om file), respectively.
  Go to the vpc_resnet50_imagenet_classification sample directory and run the following command:
```
atc --singleop=out/op_models/op_list.json --soc_version=${soc_version} --output=out/op_models
```
  - --soc_version: indicates the version of the .
    Set --soc_version to the version, which is exactly the .ini file name stored in atc/data/platform_config in the ATC installation path.
  - --output: specifies the directory for storing the generated .om file, that is, the out/op_models directory.
  For details about the available options, see ATC Instructions.
Log in to the development environment as the running user and go to the directory of the vpc_resnet50_imagenet_classification sample.
Prepare an input image.
Obtain the input images of the sample from the following links and upload the obtained images to /data under the vpc_resnet50_imagenet_classification sample directory in the development environment as the running user. If the directory does not exist, create one.

https://obs-9be7.obs.cn-east-2.myhuaweicloud.com/models/aclsample/dog1_1024_683.jpg

https://obs-9be7.obs.cn-east-2.myhuaweicloud.com/models/aclsample/dog2_1024_683.jpg

Run your app.

As the running user, upload the vpc_resnet50_imagenet_classification sample directory in the development environment to the operating environment (device), for example, $HOME/acl_dvpp_resnet50.
Log in to the operating environment (device) as the running user.
Go to the directory where the executable file main is located (for example, acl_dvpp_resnet50/out) and grant execute permission on the main file in the directory.
```
chmod +x main
```

Go to the directory where the executable file main is located (for example, acl_dvpp_resnet50/out) and run the executable file.

./main

After the command is executed successfully, the class indexes with the top confidence values are printed. In the following example, index 161 points to class ["basset", "basset hound"], and index 267 class points to ["standard poodle"].

[INFO] acl init success
[INFO] open device 0 success
[INFO] create context success
[INFO] create stream success
[INFO] dvpp init resource success
[INFO] load model ../model/resnet50_aipp.om success
[INFO] create model description success
[INFO] create model output success
[INFO] start to process picture:../data/dog1_1024_683.jpg
[INFO] Process dvpp success
[INFO] model execute success
[INFO] execute singleOp Cast success
[INFO] execute ArgMaxD success
[INFO] singleOp process success
[INFO] ---> index of classification result is 161
[INFO] start to process picture:../data/dog2_1024_683.jpg
[INFO] Process dvpp success
[INFO] model execute success
[INFO] execute singleOp Cast success
[INFO] execute ArgMaxD success
[INFO] singleOp process success
[INFO] ---> index of classification result is 267 
[INFO] Unload model success, modelId is 1
[INFO] execute sample success
[INFO] end to destroy stream 
[INFO] end to destroy context
[INFO] end to reset device is 0

Parent topic: App Samples