Concepts and Restrictions

Dynamic batch/image size

The batch size or image size is not fixed in certain scenarios. For example, as the number of detected objects is subject to change in the object detection+target recognition cascade scenario, the batch size of the target recognition input is dynamic.

• Dynamic batch size: The batch size is not determinable until inference time.
• Dynamic image size: The size (H x W) of each image is not determinable until inference time.

Dynamic dimension sizes (ND format only)

Dynamic dimensions for the ND format are useful in scenarios where input dimensions are unknown (such as the Transformer network).

Inference is performed on the same model.

AIPP (static or dynamic) and dynamic dimensions (ND format only) are mutually exclusive.

Inference is performed on the same model.

Output buffer allocation for model inference

You can allocate the memory based on the actual size of each level, or call aclmdlGetOutputSizeByIndex to obtain the memory size and then allocate the memory. (You are advised to use this method to ensure that the memory is sufficient.)

When static AIPP and dynamic image size are used at the same time

In the dynamic image size scenario, the width and height of the input image are uncertain. Therefore, when the insert_op_conf parameter of the ATC tool is used to transfer the AIPP configuration file, the cropping and padding functions cannot be enabled in the AIPP configuration file, and the values of src_image_size_w and src_image_size_h in the configuration file must be set to 0.

Both dynamic AIPP and dynamic batch size enabled

• When aclmdlCreateAIPP is called to set batchSize, batchSize must be set to the maximum batch size.
• For data nodes that require dynamic AIPP, the corresponding input buffer must be allocated based on the maximum batch size.

When dynamic AIPP and dynamic image size are used at the same time

• If the image cropping, resizing, or padding function is enabled in setting the dynamic AIPP attributes, dynamic image size becomes unavailable.
• If the image cropping, resizing, or padding function is not enabled when the dynamic AIPP parameters are set, ensure that the width and height configured by calling aclmdlSetAIPPSrcImageSize are the same as those configured by calling aclmdlSetDynamicHWSize. Both of them must be set to the width and height of the maximum dynamic image size during model conversion.
• For data nodes that require dynamic AIPP, the corresponding input buffer must be allocated based on the allowed maximum image size.

Dynamic AIPP and dynamic shape input (setting the shape range) are used at the same time.

The width and height of the output image of dynamic AIPP must be within the configured shape range.