--dynamic_dims

Applicability

Description

Sets dynamic dimension profiles in ND format. Applies to the scenario where the dimensions for inference are unfixed.

To support scenarios such as Transformer using dynamic dimensions, this option allows you to set dynamic dimension profiles in ND format. ND: Any format.

See Also

Use this option in conjunction with --input_shape and --input_format. This option is mutually exclusive with --dynamic_batch_size, --dynamic_image_size, --input_shape_range, and --insert_op_conf.

Argument

Argument: Formatted as "dim1,dim2,dim3;dim4,dim5,dim6;dim7,dim8,dim9"

Format: Enclose the whole option in double quotation marks (""), separate the profiles by semicolons (;), and separate values within each profile by commas (,). The dimension size values match the -1 placeholders in the --input_shape option with ordering preserved, and the number of -1 placeholders in --input_shape equals the number of dimension sizes of each profile.

Restrictions:

• For the following products, the profile range is (1, 100]. That is, at least two profiles must be set, and a maximum of 100 profiles are supported. Three to four profiles are recommended.

  Atlas A3 training product / Atlas A3 inference product

  Atlas A2 training product / Atlas A2 inference product

  Atlas 200I/500 A2 inference product

  Atlas inference product

  Atlas training product

• For the Atlas 350 Accelerator Card, the profile range is (1, 256]. That is, at least two profiles must be set, and a maximum of 100 profiles are supported. Three to four profiles are recommended.

Suggestions and Benefits

None

Example

• If the network model has only one input:

  The dimension size values match the -1 placeholders in the --input_shape argument with ordering preserved, and the number of -1 placeholders equals the number of dimension sizes of each profile. For example:

  If the ATC arguments are as follows:

  --input_shape="data:1,-1"  --dynamic_dims="4;8;16;64" --input_format=ND

  The shapes of the data operator supported by the ATC during model build are 1,4; 1,8; 1,16; 1,64.

  If the ATC arguments are as follows:

  --input_shape="data:1,-1,-1"  --dynamic_dims="1,2;3,4;5,6;7,8" --input_format=ND

  The shapes of the data operator supported by the ATC during model build are 1,1,2; 1,3,4; 1,5,6; 1,7,8

• If the network model has multiple inputs:

  The dimension values match the -1 placeholders in the --input_shape argument with ordering preserved, and the number of -1 placeholders equals the number of dimensions of each profile. Assume that a network model has three inputs: data (1, 1, 40, T), label (1, T), and mask (T, T), where T indicates a dynamic dimension. A configuration example is as follows:

  --input_shape="data:1,1,40,-1;label:1,-1;mask:-1,-1"  --dynamic_dims="20,20,1,1;40,40,2,2;80,60,4,4" --input_format=ND

  The input supports the following shape profiles at ATC build time:

  Profile 0: data(1,1,40,20)+label(1,20)+mask(1,1)

  Profile 1: data(1,1,40,40)+label(1,40)+mask(2,2)

  Profile 2: data(1,1,40,80)+label(1,60)+mask(4,4)

Dependencies and Restrictions

• Restrictions:

  Networks that contain dynamic-shape operators (middle layers of the network with unfixed shape) are not supported.

• API restrictions:

  If this option is used to set the dynamic dimensions during model conversion, you need to perform the following operations before calling the model execution APIs to run an application project for inference:

  • Use the aclmdlSetInputDynamicDims API to set the real dimensions.
  • If aclmdlSetInputDynamicDims is not called, the maximum value within the dynamic dimension range is assigned by default during model execution.

  For details about the API, see aclmdlSetInputDynamicDims.