What Is AIPP?
This section describes the definition, classification, and features of Artificial Intelligence Pre-Processing (AIPP).
AIPP is introduced for AI Core–based data preprocessing including image resize, CSC, and mean subtraction and factor multiplication (for pixel changing), prior to model inference.
Similar to DVPP, AIPP can process media data. Some functions of DVPP and AIPP are the same, for example, image resizing and format conversion. However, some functions are different. For example, DVPP can encode and decode images and videos, while AIPP supports normalization. Unlike DVPP, AIPP is used to preprocess data on the AI Core. DVPP is an embedded image processing unit of the Ascend AI Processor and provides powerful hardware acceleration for media data processing using AscendCL APIs.
AIPP and DVPP can be used separately or together. In combined applications, DVPP is used first to decode, crop, and resize images or videos. However, due to DVPP hardware restrictions, the image format and resolution after DVPP may not meet the model requirements. Therefore, AIPP is required to further perform CSC, image cropping, and border making.
If you want to get the desired image format for inference, you can use CSC. To get fixed-size images, you can use the cropping and padding functions provided by AIPP. AIPP supports two modes: static and dynamic. The following discusses the differences between the two modes.
Static AIPP/Dynamic AIPP
Static AIPP mode and dynamic AIPP mode are mutually exclusive. You must select between the two modes by setting aipp_mode. Go to AIPP Configuration Sample for a configuration example, or Configuration File Template for more about the parameter.
- Static AIPP: During model conversion, set the AIPP mode to static and specify the AIPP attributes. After the model is generated, the AIPP attribute values are saved in the offline model. Fixed AIPP configurations (unmodifiable at inference time) are used in every model inference.
In static AIPP mode, the same set of AIPP configurations is used across batches.
- Dynamic AIPP: During model conversion, set the AIPP mode to dynamic and specify different sets of dynamic AIPP configurations as required, so that different sets of AIPP configurations can be used for model inference. In this mode, the AIPP configurations will be tailored for different service scenarios. For example, different normalization parameters can be set for different cameras, or the support for both the YUV420 and RGB input formats can be added.
If dynamic AIPP is used in model conversion, the aclmdlSetInputAIPP API should be called before the AscendCL aclmdlExecute API to set the runtime dynamic AIPP parameters. For details about the APIs, see Model Execution.
In dynamic AIPP mode, different configurations (for example, different crop parameters) are used across batches, as defined by the dynamic AIPP struct. For details about the dynamic AIPP struct, see Input Struct.
AIPP supports the following input image formats: YUV420SP_U8, RGB888_U8, XRGB8888_U8, and YUV400_U8.
CSC
CSC reformats images according to the model requirements. Set csc_switch to enable or disable CSC. For details, see Configuration File Template.
As long as you determine the input and output image formats for AIPP, the CSC parameters are also determined. You can directly use the corresponding template provided in this document. For details, see CSC Configuration.
Image Resize
Image resize involves cropping and padding. You can find corresponding parameters in the configuration template. For details about parameter description, see Configuration File Template.
For details about this function and AIPP parameter settings, see Cropping and Padding Configuration.