--dynamic_dims

Description

Sets dynamic dimension profiles in ND format. Applies to the scenario where the dimensions for inference are unfixed.

To support scenarios such as Transformer using dynamic dimensions, this option allows you to set dynamic dimension profiles in ND format. ND: Any format.

See Also

• Use this option in conjunction with --input_shape and --input_format. This option is mutually exclusive with --dynamic_batch_size, --dynamic_image_size, --input_shape_range, and --insert_op_conf.
• This parameter cannot be used together with --framework = 1. The *.air model does not support dynamic shape profiles.

Argument

Argument: Formatted as "dim1,dim2,dim3;dim4,dim5,dim6;dim7,dim8,dim9"

Format: Enclose the whole argument in double quotation marks (""), and separate the profiles by commas (,). The dimension size values match the -1 placeholders in the --input_shape argument with ordering preserved, and the number of -1 placeholders equals the number of dimension sizes of each profile.

Restrictions:

• The profile range is (1,100]. That is, at least two profiles must be set, and a maximum of 100 profiles are supported. Three to four profiles are recommended.

Suggestions and Benefits

None

Example

• If the network model has only one input:

  The dimension size values match the -1 placeholders in the --input_shape argument with ordering preserved, and the number of -1 placeholders equals the number of dimension sizes of each profile. For example:

  If the ATC arguments are as follows:

  --input_shape="data:1,-1"  --dynamic_dims="4;8;16;64" --input_format=ND

  The shapes of the data operator supported by the ATC during model build are 1,4; 1,8; 1,16; 1,64.

  If the ATC arguments are as follows:

  --input_shape="data:1,-1,-1"  --dynamic_dims="1,2;3,4;5,6;7,8" --input_format=ND

  The shapes of the data operator supported by the ATC during model build are 1,1,2; 1,3,4; 1,5,6; 1,7,8

• If the network model has multiple inputs:

  The dimension values match the -1 placeholders in the --input_shape argument with ordering preserved, and the number of -1 placeholders equals the number of dimensions of each profile. Assume that a network model has three inputs: data (1, 1, 40, T), label (1, T) and mask (T, T), where T indicates a dynamic dimension. The configuration example is as follows:

  --input_shape="data:1,1,40,-1;label:1,-1;mask:-1,-1"  --dynamic_dims="20,20,1,1;40,40,2,2;80,60,4,4" --input_format=ND

  The input supports the following shape profiles at ATC build time:

  Profile 0: data(1,1,40,20)+label(1,20)+mask(1,1)

  Profile 1: data(1,1,40,40)+label(1,40)+mask(2,2)

  Profile 2: data(1,1,40,80)+label(1,60)+mask(4,4)

Applicability

Atlas 200/300/500 Inference Product

Atlas Training Series Product

Dependencies and Restrictions

• Restrictions:

  Networks that contain dynamic-shape operators (middle layers of the network with unfixed shape) are not supported.

• API restrictions:

  If this option is used to set the dynamic dimensions during model conversion, you need to perform the following operations before calling the model execution APIs to run an application project for inference:

  • Use the aclmdlSetInputDynamicDims API provided by AscendCL to set the runtime dimensions.
  • If aclmdlSetInputDynamicDims is not called, the maximum value within the dynamic dimension range is assigned by default during model execution.

  For details about the APIs, see aclmdlSetInputDynamicDims.