--dynamic_dims
Applicable Products
All processors
Description
Sets dynamic dimension profiles in ND format. Applies to the scenario where the dimensions for inference are unfixed.
To support scenarios such as Transformer using dynamic dimensions, this option allows you to set dynamic dimension profiles in ND format. ND: Any format.
See Also
- Use this option together with --input_shape and --input_format. This option is mutually exclusive with --dynamic_batch_size, --dynamic_image_size, --input_shape_range, and --insert_op_conf.
Arguments
Argument: Formatted as "dim1,dim2,dim3;dim4,dim5,dim6;dim7,dim8,dim9"
Format: Enclose all profiles in double quotation marks (""), separate profiles by semicolons (;), and separate values within each profile by commas (,). The dimension size values in each profile map sequentially to the -1 placeholders in the --input_shape, and the number of -1 placeholders in --input_shape equals the number of dimension sizes of each profile.
Restrictions:
- For the following products, at least two profiles must be set, and a maximum of 100 profiles are supported, that is, (1,100]. Three to four profiles are recommended.
Atlas A3 training products /Atlas A3 inference products Atlas A2 training products /Atlas A2 inference products Atlas 200I/500 A2 inference products Atlas inference products Atlas training products - For the
Ascend 950PR /Ascend 950DT , at least two profiles must be set, and a maximum of 256 profiles are supported, that is, (1,256]. Three to four profiles are recommended.
Suggestions and Benefits
None
Example
- If the network model has only one input:
The dimension size values match the -1 placeholders in the --input_shape argument with ordering preserved, and the number of -1 placeholders equals the number of dimension sizes of each profile. For example:
If the ATC arguments are as follows:
--input_shape="data:1,-1" --dynamic_dims="4;8;16;64" --input_format=ND
The shapes of the data operator supported by the ATC during model build are 1,4; 1,8; 1,16; 1,64.
If the ATC arguments are as follows:
--input_shape="data:1,-1,-1" --dynamic_dims="1,2;3,4;5,6;7,8" --input_format=ND
The shapes of the data operator supported by the ATC during model build are 1,1,2; 1,3,4; 1,5,6; 1,7,8
- If the network model has multiple inputs:
The dimension values match the -1 placeholders in the --input_shape argument with ordering preserved, and the number of -1 placeholders equals the number of dimensions of each profile. Assume that a network model has three inputs: data (1, 1, 40, T), label (1, T), and mask (T, T), where T indicates a dynamic dimension. The configuration example is as follows:
--input_shape="data:1,1,40,-1;label:1,-1;mask:-1,-1" --dynamic_dims="20,20,1,1;40,40,2,2;80,60,4,4" --input_format=ND
The input supports the following shape profiles at ATC build time:
Profile 0: data(1,1,40,20)+label(1,20)+mask(1,1)
Profile 1: data(1,1,40,40)+label(1,40)+mask(2,2)
Profile 2: data(1,1,40,80)+label(1,60)+mask(4,4)
Dependencies and Restrictions
- Option usage:
Networks that contain dynamic-shape operators (middle layers of the network with unfixed shape) are not supported.
- API usage:
If this option is used to set the dynamic dimensions during model conversion, you need to perform the following operations before calling the model execution APIs to run an application project for inference:
- Use the aclmdlSetInputDynamicDims API to set the actual dimension values.
- If aclmdlSetInputDynamicDims is not called, the maximum value within the dynamic dimension range is assigned by default during model execution.