Concepts and Restrictions

Dynamic batch/image size

The batch size or image size is not fixed in certain scenarios. For example, in the object detection+target recognition cascade scenario where the number of detected objects is subject to change, the batch size of the target recognition input is dynamic.

• Dynamic batch size: The batch size is not determinable until inference time.
• Dynamic image size: The image size (H x W) of each image is not determinable until inference time.

Dynamic dimensions (ND format)

Dynamic dimensions for the ND format are useful in scenarios where input dimensions are unknown (such as the Transformer network).

Performing inference on a single model

AIPP (static or dynamic) and dynamic dimensions (ND format only) are mutually exclusive.

Performing inference on a single model

Output buffer allocation for model inference

You can allocate buffer based on the runtime dimensions. To be accurate, you can call aclmdlGetOutputSizeByIndex to obtain the required buffer size before allocating the buffer.

Both static AIPP and dynamic image size enabled

Because the width and height of the input image are uncertain in the dynamic image size scenario, when the insert_op_conf parameter of the ATC tool is used to take the AIPP configuration file, the crop and padding functions cannot be enabled in the AIPP configuration file. Additionally, the src_image_size_w and src_image_size_h values in the configuration file need to be set to 0.

Both dynamic AIPP and dynamic batch size enabled

• The batchSize parameter passed to the aclmdlCreateAIPP call must be set to the allowed maximum profile.
• For data nodes that require dynamic AIPP, the input buffer must be allocated based on the maximum allowed batch size.

Both dynamic AIPP and dynamic image size enabled

• If the image cropping, resizing, or padding function is enabled in setting the dynamic AIPP attributes, dynamic image size becomes unavailable.
• If the image cropping, resizing, or padding function is not enabled in setting the dynamic AIPP attributes, and dynamic AIPP and dynamic image size are both available, ensure that the maximum allowed image size profile is passed to the aclmdlSetAIPPSrcImageSize call and aclmdlSetDynamicHWSize call.
• For data nodes that require dynamic AIPP, the input buffer must be allocated based on the maximum allowed image size.

Dynamic AIPP and dynamic shape input (setting the shape range) used at the same time

The width and height of the output image of dynamic AIPP must be within the configured shape range.