Documentation

Configuration of the Dynamic-Shape Single-Operator Description File

In dynamic shape scenarios, the content of a single-operator description file varies according to the scenario. This section provides configuration examples in different scenarios.

  • The shape is not specified during model build. During model execution, the output shape can be inferred based on the input static shape.
     1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
    [
  {
    "op": "Add",
    "name": "add",
    "input_desc": [
      {
        "format": "ND",
        "shape": [-1,16],
        "shape_range": [[0, 32]],
        "type": "int64"
      },
      {
        "format": "ND",
        "shape": [-1,16],
        "shape_range": [[0, 32]],
        "type": "int64"
      }
    ],
    "output_desc": [
      {
        "format": "ND",
        "shape": [-1,16],
        "shape_range": [[0,32]],
        "type": "int64"
      }
    ]
  }
]
  • The shape is not specified during model build. During model execution, the output shape can be inferred based on the input static shape and constant.
     1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
    [
  {
    "op": "TopK",
    "name": "topK",
    "input_desc": [
      {
        "format": "ND",
        "shape": [-1],
       "shape_range": [[1,-1]],
        "type": "int32"
      },
      {
        "format": "ND",
        "shape": [],      # Constant values are passed during inference.
        "type": "int32"
      }
    ],
    "output_desc": [
      {
        "format": "ND",
        "shape": [-1],
        "shape_range": [[1,-1]],
        "type": "int32"
      },
	  {
        "format": "ND",
        "shape": [-1],
        "shape_range": [[1,-1]],
        "type": "int32"
      }],
      "attr": [
      {
        "name": "sorted",
        "type": "bool",
        "value": true
      }
    ]
  }
]
  • The shape is not specified during model build. During model execution, the output shape cannot be inferred based on the input static shape, but the output shape range can be obtained.
    In this scenario, the dynamic shape in TensorDesc is –1 in the output parameter output_desc, and the corresponding shape_range is provided.
     1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
    [
  {
    "op": "Where",
    "name": "where",
    "input_desc": [
      {
        "format": "ND",
        "shape": [-1],
        "shape_range": [[1,-1]],
        "type": "int32"
      }
    ],
    "output_desc": [
      {
        "format": "ND",
        "shape": [-1, 1],
        "shape_range": [[1,-1]],
        "type": "int64"
      }
    ]
  }
]