Dynamic Shape Configuration

Function Description

Configuring dynamic shapes provides more shape range information during graph compilation. It enables the compiler to compile operators efficiently and manage memory better. The maxOutputSize parameter is used to configure a more accurate maximum output value. The graph executor allocates the output memory before graph calculation based on this configuration to improve performance.

Constraints

None

How to Use

Assume that there is a resize sink map of a dynamic shape. The sink map has two inputs: one is the image data to be resized, and the other is the target size information.

1
2

def resize_bilinear(x: torch.Tensor, hw: torch.Tensor):
      return F.interpolate(x, size=[hw[0], hw[1]], mode="bilinear", align_corners=True)

For the dynamic sink map, the specific configuration is as follows:

• ge.inputShape: shape range of the dynamic input.
  • To set the shape range based on node names, the format is "input_name1:n1,c1,h1,w1;input_name2:n2,c2,h2,w2", for example, "input_name1:8~20,3,5,-1;input_name2:5,3~9,10,-1". Enclose all the nodes in double quotation marks (""), and separate them by semicolons (;). input_name must be the node name in the network model before model conversion. As a best practice, you should set the parameter based on data node names.
  • To set the shape range based on node indexes, the format is "n1,c1,h1,w1;n2,c2,h2,w2", for example, "8~20,3,5,-1;5,3~9,10,-1". If the node name is not specified, the nodes are sorted by the index and separated by semicolons (;). When the shape range is specified based on the index, data nodes must be configured with the index attribute (starting from 0) to indicate their input sequence.
  • A tilde (~) denotes the value range of a dynamic dimension; a fixed number stands for a static dimension; -1 represents a dynamic dimension without a specified value range.
• ge.outputMaxSize: (optional) maximum memory of the graph output.
  • If this parameter is set, the dataflow framework uses it to allocate memory for the sink map output in advance, eliminating one round of data copy and boosting execution performance.

    If the maximum output shape is [1,3,2567,3440] and dtype is float32 (4 bytes for fp32 storage), the value of outputMaxSize is calculated as 1 × 3 × 2567 × 3440 × 4 = 105965760.

    An error will be thrown if outputMaxSize is set to a value smaller than the actual output size.

  • If this parameter is not set, the output memory will be allocated according to actual requirements after model execution.
• inputs_tensor_desc
  • data_type: data type of the tensor.
  • shape: shape information of the tensor. Set this parameter to -1 for dynamic dimensions.
  • format: Each input format must be set in line with service requirements. The NCHW, NHWC, and ND formats are supported.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18

{
    "build_options": {
        "ge.inputShape": "1,3,1~1728,1~1728;2",
        "ge.outputMaxSize": "107495424"
    },
    "inputs_tensor_desc": [
      {
        "data_type": "DT_FLOAT",
        "shape": [1, 3, -1, -1],
        "format": "NCHW"
      },
      {
        "data_type": "DT_INT32",
        "shape": [2],
        "format": "ND"
      }
    ]
}