Dynamic Shape Concepts and Restrictions

Dynamic batch/image size

The batch size or image size is not fixed in certain scenarios. For example, in the object detection+target recognition cascade scenario where the number of detected objects is subject to change, the batch size of the target recognition input is dynamic.

• Dynamic batch size: The batch size is not determinable during inference.
• Dynamic image size: The size (H × W) of each image is not determinable during inference.

Dynamic dimension sizes (ND format only)

Dynamic dimensions for the ND format are useful in scenarios where input dimensions are unknown (such as the Transformer network).

Performing inference on a single model

AIPP (static or dynamic) and dynamic dimensions (ND format only) are mutually exclusive.

Performing inference on a single model

Output buffer allocation for model inference

You can allocate buffer based on the runtime dimensions. To be accurate, you can call aclmdlGetOutputSizeByIndex to obtain the required buffer size before allocating the buffer.

Both static AIPP and dynamic image size enabled

Because the width and height of the input image are uncertain in the dynamic image size scenario, when the insert_op_conf parameter of the ATC tool is used to take the AIPP configuration file, the cropping and padding functions cannot be enabled in the AIPP configuration file. Additionally, the src_image_size_w and src_image_size_h parameters in the configuration file need to be set to 0.

Both dynamic AIPP and dynamic batch size enabled

• The batchSize parameter passed to the aclmdlCreateAIPP API must be set to the allowed maximum profile.
• For data nodes that require dynamic AIPP, the corresponding input buffer must be allocated based on the allowed maximum batch size.

Both dynamic AIPP and dynamic image size enabled

• If the image cropping, resizing, or padding function is enabled in setting the dynamic AIPP attributes, dynamic image size becomes unavailable.
• If the image cropping, resizing, or padding function is not enabled when setting the dynamic AIPP parameters, and dynamic AIPP and dynamic image size are both available, ensure that the allowed maximum image size profile is passed to both the aclmdlSetAIPPSrcImageSize and aclmdlSetDynamicHWSize APIs.
• For data nodes that require dynamic AIPP, the corresponding input buffer must be allocated based on the allowed maximum image size.

Dynamic AIPP and dynamic shape input (setting the shape range) used at the same time

The width and height of the output image of dynamic AIPP must be within the configured shape range.