AIPP

Overview

The AI pre-processing (AIPP) module is introduced for image preprocessing on AI Cores, including color space conversion (CSC), image normalization (by subtracting the mean value or multiplying a factor), image cropping (by specifying the crop start and cropping the image to the size required by the neural network). AIPP is classified into static AIPP and dynamic AIPP. They are mutually exclusive and cannot be both set.

• Static AIPP: At model build time, specify the AIPP mode to static and set the AIPP parameters. After the model is generated, the AIPP parameter values are saved in the offline model. The same AIPP parameter configurations are used in each model inference phase and are unmodifiable.
• Dynamic AIPP: At model build time, specify the AIPP mode to dynamic. Each time before you run the model for inference, set the AIPP configurations as needed to implement application-specific AIPP preprocessing. In dynamic AIPP mode, the preprocessing settings vary according to the service requirements in the scenario where, for example, cameras use different normalization parameters or both YUV420 and RGB input formats need to be supported.

For details about the AIPP function, see ATC Instructions.

Procedure

The following briefly describes how to support AIPP at model build time.

1. Keep the channel size of the Data operator input the same as the channel count of the image.

   1 2 3 4 5

   auto shape_data = vector<int64_t>({ 1,10,12,12 }); TensorDesc desc_data(ge::Shape(shape_data), FORMAT_NCHW, DT_FLOAT16); auto data = op::Data("data"); data.update_input_desc_x(desc_data); data.update_output_desc_y(desc_data);

2. Prepare the AIPP configuration file by referring to ATC Instructions. The following are configuration examples.
   The following is an example of the static AIPP configuration file.

   1 2 3 4 5 6 7 8

   aipp_op { aipp_mode:static # Static AIPP input_format:YUV420SP_U8 csc_switch:true var_reci_chn_0:0.00392157 var_reci_chn_1:0.00392157 var_reci_chn_2:0.00392157 }

   The following is an example of the dynamic AIPP configuration file. (Only the following fields need to be configured.)

   1 2 3 4 5

   aipp_op { aipp_mode:dynamic # Dynamic AIPP related_input_rank: 0 # Optional. It indicates the AIPP processing start among the inputs. The default value is 0, which indicates that AIPP processing starts from input0. max_src_image_size:752640 # The maximum size of the input image. This parameter is required in dynamic AIPP. }

3. At model build time, set INSERT_OP_FILE and INPUT_FORMAT by configuring options in the aclgrphBuildModel call.

   1 2 3 4 5 6 7

   void PrepareOptions(std::map<AscendString, AscendString>& options) { options.insert({ {ge::ir_option::EXEC_DISABLE_REUSED_MEMORY, "1"}, {ge::ir_option::INSERT_OP_FILE, PATH + "aipp_nv12_img.cfg"}, {ge::ir_option::INPUT_FORMAT, "NCHW"} }); }

4. Build an offline model with the Aipp operator automatically inserted.

   

Precautions

• If you have configured static AIPP with the dynamic shape range specified by INPUT_SHAPE:

  If the model has only one input, this scenario is not supported. If the model has multiple inputs, you must configure different input nodes accordingly, such as setting one input node with static AIPP and another node with a dynamic shape.

• If you have configured static AIPP and set the dynamic image size (unfixed width and height of each input image) using DYNAMIC_BATCH_SIZE:

  The cropping and padding functions cannot be enabled in the AIPP configuration file, and the values of src_image_size_w and src_image_size_h must be set to 0 in the configuration file.

• If you have configured dynamic AIPP with the dynamic shape range specified by INPUT_SHAPE, the width and height of the AIPP output must be within the range specified by INPUT_SHAPE.
• If you have configured dynamic AIPP and set the dynamic batch size using DYNAMIC_BATCH_SIZE:

  In your inference code, call the aclmdlSetInputAIPP API provided by AscendCL to set dynamic AIPP parameters. Ensure that batchSize is set to the maximum batch size. For details about the APIs, see aclmdlSetInputAIPP.

• If you have configured dynamic AIPP and set the dynamic image size (unfixed width and height of each input image) using DYNAMIC_IMAGE_SIZE:

  In your inference code, call the aclmdlSetInputAIPP API provided by AscendCL to set dynamic AIPP parameters. Ensure that cropping and padding are disabled. In this scenario, ensure that the width and height configured by calling aclmdlSetInputAIPP are the same as those configured by calling aclmdlSetDynamicHWSize. That is, the width and height must be set to the maximum dynamic image size. For details about the APIs, see Model Execution.