Dynamic Shape

INPUT_SHAPE

Sets the input shape of a model. The corresponding options parameter is ge.inputShape.

Parameter values:

If the model uses a static shape, INPUT_SHAPE is optional.
- If the model uses a single input, the shape information is "input_name:n,c,h,w".
- If the model has multiple inputs, the shape information is "input_name1:n1,c1,h1,w1;input_name2:n2,c2,h2,w2". Different inputs are separated by semicolons (;). input_name must be the name of a node in the network model before conversion.
If the model uses a non-static shape, INPUT_SHAPE must be configured.
If one or more dimension values of the input data in the original model are not fixed, the model can be converted by setting the shape profile or shape range.
- Setting the shape profile (static shape): The shape profiles include the batch size, image size, and specified dynamic dimension profiles.
  When setting the INPUT_SHAPE parameter, set the corresponding dimension value to -1 and use DYNAMIC_BATCH_SIZE (setting batch size profiles), DYNAMIC_IMAGE_SIZE (setting image size profiles), or DYNAMIC_DIMS (setting dynamic dimension profiles). For details, see the parameter description of DYNAMIC_BATCH_SIZE, DYNAMIC_IMAGE_SIZE, and DYNAMIC_DIMS.
- Setting the shape range (dynamic shape)
  For Atlas 200I/500 A2 inference products , the shape range cannot be set.
  When setting the INPUT_SHAPE parameter, you can set the value of the corresponding dimension to a range, for example, 1–10. The specified range must be valid.
  - To set the shape range based on node names, the format is "input_name1:n1,c1,h1,w1;input_name2:n2,c2,h2,w2", for example, "input_name1:8~20,3,5,-1;input_name2:5,3~9,10,-1". The specified nodes must be enclosed in double quotation marks ("") and separated by semicolons (;). input_name must be the node name in the network model before conversion. As a best practice, you should set the parameter based on node names.
  - To set the shape range based on node indexes, the format is "n1,c1,h1,w1;n2,c2,h2,w2", for example, "8~20,3,5,-1;5,3~9,10,-1". If the node name is not specified, the nodes are sorted by the index and separated by semicolons (;). When the shape range is specified based on the index, the index attribute must be set sequentially from 0 for data nodes.
  If you do not want to specify the dimension range or value, you can set it to -1, indicating that the dimension can be any value greater than or equal to 0. In this scenario, the upper limit of the value is the int64 type range. However, the value is limited by the size of the physical memory on the host and device, so you can increase the memory size to support it.
Scalar model shape:
- Non-dynamic profile scenario:
  Shape is a scalar input, which is optional. For example, if the model has two inputs (input_name1 is a scalar with shape in the "[]" format, and input_name2 has the shape of [n2,c2,h2,w2]), the shape information of the model is "input_name1:;input_name2:n2,c2,h2,w2". Different inputs are separated by semicolons (;). input_name must be the node name in the network model before conversion. If the scalar input needs to be configured, leave it empty.
- Dynamic profile scenario:
  If the model input has both scalar shape and dynamic-profile shape, the scalar input must be configured. For example, if a model has three inputs: A:[-1,c1,h1,w1], B:[], and C:[n2,c2,h2,w2], the shape information is "A:-1,c1,h1,w1;B:;C:n2,c2,h2,w2". Scalar input B must be configured.

Configuration example:

Static shape. For example, if the input shape information of a network consists of input 1 (input_0_0 [16,32,208,208]) and input 2 (input_1_0 [16,64,208,208]), the configuration of INPUT_SHAPE is as follows:
```
{ge::ir_option::INPUT_SHAPE, "input_0_0:16,32,208,208;input_1_0:16,64,208,208"}
```
Non-static shape, static shape:
- For details about how to set the batch size, see DYNAMIC_BATCH_SIZE.
- For details about how to set the image size, see DYNAMIC_IMAGE_SIZE.
- For details about how to set profiles for a specified dimension, see DYNAMIC_DIMS.
Non-static shape, dynamic shape (shape range example):
```
{ge::ir_option::INPUT_SHAPE, "input_0_0:1~10,32,208,208;input_1_0:16,64,100~208,100~208"}
```
For details about the examples and precautions, see "Additional Features" > "Shape Range of Dynamic Input".
Scalar shape
- Non-dynamic profile scenario:
  Shape is a scalar input, which is optional. For example, if the model has two inputs—input_name1 is a scalar and input_name2 has the shape of [16,32,208,208], the configuration example is as follows:
```
{ge::ir_option::INPUT_SHAPE, "input_name1:;input_name2:16,32,208,208"}
```
  In the preceding example, input_name1 is optional.
- Dynamic profile scenario:
  Shape is a scalar input, which is mandatory. For example, if the model has three inputs and the shape information is A:[-1,32,208,208], B:[], and C:[16,64,208,208], the configuration example is as follows (A is the dynamic profile input, and the batch size profile is used):
```
{ge::ir_option::INPUT_SHAPE, "A:-1,32,208,208;B:;C:16,64,208,208"}, 
{ge::ir_option::DYNAMIC_BATCH_SIZE, "1,2,4"} 
```

If this parameter is used to set the shape range during model conversion, call aclmdlSetDatasetTensorDesc to set the actual input tensor description (input shape range) before model execution during model inference. After model execution, call aclmdlGetDatasetTensorDesc to obtain the tensor description dynamically output by the model. Then, call the APIs under aclTensorDesc to obtain the memory size occupied by the output tensor data, tensor format, and tensor dimensions.

For details about how to use aclmdlSetDatasetTensorDesc and aclmdlGetDatasetTensorDesc, see GE APIs.

Applicable products:

All processors

DYNAMIC_BATCH_SIZE

Sets the dynamic batch size profile. This parameter applies to the scenario where the number of images processed per inference batch is unfixed. The corresponding options parameter is ge.dynamicBatchSize.

This parameter must be used together with INPUT_SHAPE and is mutually exclusive with DYNAMIC_IMAGE_SIZE and DYNAMIC_DIMS. In addition, N must be in the first place of the shape. The first place of the shape must be set to -1. If N is not in the first place, use DYNAMIC_DIMS to set it.

Parameter values: batch size profiles, for example, "1,2,4,8".

Format: Enclose the specified parameter in double quotation marks ("") and separate the batch sizes with commas (,).

Restrictions:

For the following products, the batch size profile range is (1, 100]. At least two profiles must be set, and a maximum of 100 profiles are supported. The recommended value range for each profile is [1, 2048].
Atlas A3 training products / Atlas A3 inference products

Atlas A2 training products / Atlas A2 inference products

Atlas 200I/500 A2 inference products

Atlas inference products

Atlas training products
For the Ascend 950PR / Ascend 950DT , the batch size profile range is (1, 256]. That is, at least two profiles must be set and a maximum of 256 profiles are supported. The recommended value range for each profile is [1, 2048].

Configuration example:

The value -1 of INPUT_SHAPE indicates that the dynamic batch size is enabled.

{ge::ir_option::INPUT_FORMAT, "NHWC"},
{ge::ir_option::INPUT_SHAPE, "data:-1,3,416,416"}, 
{ge::ir_option::DYNAMIC_BATCH_SIZE, "1,2,4,8"}

For details about the examples and precautions, see "Additional Features" > "Dynamic Batch Size".

Applicable products:

All processors

DYNAMIC_IMAGE_SIZE

Dynamic input image size. Applies to the scenario where the resolution of images input for inference is unfixed. The corresponding options parameter is ge.dynamicImageSize.

This parameter must be used together with INPUT_SHAPE and is mutually exclusive with DYNAMIC_BATCH_SIZE and DYNAMIC_DIMS.

Parameter values: "imagesize1_height,imagesize1_width;imagesize2_height,imagesize2_width"

Format: Enclose the specified parameter in double quotation marks ("") and separate profiles by semicolons (;) and values within each profile by commas (,).

Restrictions:

For the following products, the profile range is (1, 100]. At least two profiles must be set, and a maximum of 100 profiles are supported.
Atlas A3 training products / Atlas A3 inference products

Atlas A2 training products / Atlas A2 inference products

Atlas 200I/500 A2 inference products

Atlas inference products

Atlas training products
For the Ascend 950PR / Ascend 950DT , the profile range is (1, 256]. At least two profiles must be set, and a maximum of 256 profiles are supported.

Configuration example:

The value -1 of INPUT_SHAPE indicates that the dynamic image size is enabled.

{ge::ir_option::INPUT_FORMAT, "NCHW"}, 
{ge::ir_option::INPUT_SHAPE, "data:8,3,-1,-1"}, 
{ge::ir_option::DYNAMIC_IMAGE_SIZE, "416,416;832,832"}

For details about the examples and precautions, see "Additional Features" > "Dynamic Image Size".

Applicable products:

All processors

DYNAMIC_DIMS

Sets dynamic dimension profiles in ND format. This parameter applies to the scenario where the dimensions for inference are unfixed. The corresponding options parameter is ge.dynamicDims.

This parameter must be used together with INPUT_SHAPE and is mutually exclusive with DYNAMIC_BATCH_SIZE and DYNAMIC_IMAGE_SIZE.

Parameter value: Set it in the "dim1,dim2,dim3;dim4,dim5,dim6;dim7,dim8,dim9" format.

Format: Enclose all profiles in double quotation marks (""), separate the profiles by semicolons (;), and separate values within each profile by commas (,). The dimension size values match the -1 placeholders in the INPUT_SHAPE parameter with ordering preserved, and the number of -1 placeholders equals the number of dimension sizes of each profile.

Restrictions:

For the following products, the profile range is (1, 100]. At least two profiles must be set, and a maximum of 100 profiles are supported. Three to four profiles are recommended.
Atlas A3 training products / Atlas A3 inference products

Atlas A2 training products / Atlas A2 inference products

Atlas 200I/500 A2 inference products

Atlas inference products

Atlas training products
For the Ascend 950PR / Ascend 950DT , the profile range is (1, 256]. At least two profiles must be set, and a maximum of 256 profiles are supported. Three to four profiles are recommended.

Configuration example:

{ge::ir_option::INPUT_FORMAT, "ND"},
{ge::ir_option::INPUT_SHAPE, "data:1,-1"}, 
{ge::ir_option::DYNAMIC_DIMS, "4;8;16;64"}  
// During model compilation, the supported shape of the data operator is 1,4; 1,8; 1,16;1,64.
{ge::ir_option::INPUT_FORMAT, "ND"},
{ge::ir_option::INPUT_SHAPE, "data:1,-1,-1"}, 
{ge::ir_option::DYNAMIC_DIMS, "1,2;3,4;5,6;7,8"}  
// During model compilation, the supported shape of the data operator is 1,1,2; 1,3,4; 1,5,6; 1,7,8.

For details about the examples and precautions, see "Additional Features" > "Dynamic Dimension Size".

Applicable products:

All processors

AC_PARALLEL_ENABLE

Whether to allow AI CPU operators and AI Core operators to run in parallel in a dynamic shape graph. The corresponding options parameter is ac_parallel_enable.

In a dynamic shape graph, when it is enabled, the system automatically identifies AI CPU operators that can be concurrently executed with the AI Core operators in the graph. Operators of different engines are distributed to different flows to implement parallel execution among multiple engines, improving resource utilization and dynamic shape execution performance.

Arguments:

1: AI CPU operators and AI Core operators are allowed to run in parallel.
0 (default): AI CPU operators are not separately distributed.

Configuration example:

{ge::ir_option::AC_PARALLEL_ENABLE, "1"}

Applicable products:

Atlas inference product : supported
Atlas training product : supported
AS31XM1X AI Processor: supported
SoC (TsnsC): not supported
Atlas inference product : supported
Atlas inference product : supported

Parent topic: aclgrphBuildModel Configuration Parameters