昇腾社区首页
中文
注册

单算子描述文件配置

本章节中的单算子是基于Ascend IR定义的,描述文件为json格式。关于单算子的定义请参见算子清单

单算子生成的离线模型文件命名规则为:序号_opType_输入的描述(dataType_format_shape)_输出的描述(dataType_format_shape)。

dataType以及format对应枚举值请从${INSTALL_DIR}/include/graph/types.h文件中查看,枚举值从0开始依次递增。其中,${INSTALL_DIR}请替换为CANN软件安装后文件存储路径。例如,若安装的Ascend-cann-toolkit软件包,则安装后文件存储路径为:$HOME/Ascend/ascend-toolkit/latest。

  • format为ND:
    [
    {
      "op": "GEMM",
      "input_desc": [
        {
          "format": "ND",
          "shape": [16, 16],
          "type": "float16"
        },
        {
          "format": "ND",
          "shape": [16, 16],
          "type": "float16"
        },
        {
          "format": "ND",
          "shape": [16, 16],
          "type": "float16"
        },
        {
          "format": "ND",
          "shape": [],
          "type": "float16"
        },
        {
          "format": "ND",
          "shape": [],
          "type": "float16"
        }
      ],
      "output_desc": [
        {
          "format": "ND",
          "shape": [16, 16],
          "type": "float16"
        }
      ],
      "attr": [
      {
        "name": "transpose_a",
        "type": "bool",
        "value": false
      },
      {
        "name": "transpose_b",
        "type": "bool",
        "value": false
        }
      ]
    }
    ]

    上述单算子转换后的离线模型为:0_GEMM_1_2_16_16_1_2_16_16_1_2_16_16_1_2_1_2_1_2_16_16.om

  • format为NCHW:
    [
     {
       "op": "Conv2D",  
       "input_desc": [     
         {        
           "format": "NCHW",      
           "shape": [1, 3, 16, 16],     
           "type": "float16"     
         },      
         {       
           "format": "NCHW",       
           "shape": [3, 3, 3, 3],        
           "type": "float16"       
         }     
       ],    
       "output_desc": [     
         {     
           "format": "NCHW",        
           "shape": [1, 3, 16, 16],     
           "type": "float16"      
         }    
       ],   
       "attr": [      
           {       
             "name": "strides",       
             "type": "list_int",       
             "value": [1, 1, 1, 1]    
           },     
           {       
             "name": "pads",      
             "type": "list_int",      
             "value": [1, 1, 1, 1]     
           },      
           {        
             "name": "dilations",      
             "type": "list_int",     
             "value": [1, 1, 1, 1]    
           }  
       ]  
     } 
     ]

    上述单算子转换后的离线模型为:0_Conv2D_1_0_1_3_16_16_1_0_3_3_3_3_1_0_1_3_16_16.om

  • Tensor的实现format与原始format不同
    ATC模型转换时,会将origin_formatorigin_shape转成离线模型需要的formatshape
    [
      {
        "op": "Add",
        "input_desc": [
          {
            "format": "NC1HWC0",
            "origin_format": "NCHW",
            "shape": [8, 1, 16, 4, 16],
            "origin_shape": [8, 16, 16, 4],
            "type": "float16"
          },
          {
            "format": "NC1HWC0",
            "origin_format": "NCHW",
            "shape": [8, 1, 16, 4, 16],
            "origin_shape": [8, 16, 16, 4],
            "type": "float16"
          }
        ],
        "output_desc": [
          {
            "format": "NC1HWC0",
            "origin_format": "NCHW",
            "shape": [8, 1, 16, 4, 16],
            "origin_shape": [8, 16, 16, 4],
            "type": "float16"
          }
        ]
      }
    ]

    上述单算子转换后的离线模型为:0_Add_1_3_8_1_16_4_16_1_3_8_1_16_4_16_1_3_8_1_16_4_16.om

  • 输入指定为常量

    该场景下,支持设置为常量的输入,新增is_constconst_value两个参数,分别表示是否为常量以及常量取值,const_value当前仅支持一维list配置,具体配置个数由shape取值决定,例如,如下样例中shape为2,则const_value中列表个数为2;const_value中取值类型由type决定,假设type取值为float16,则单算子编译时会自动将const_value中的取值转换为float16格式的取值。

    [
      {
        "op": "ResizeBilinearV2",
        "input_desc": [
          {
            "format": "NHWC",
            "name": "x",
            "shape": [
              4,
              16,
              16,
              16
            ],
            "type": "float16"
          },
          {
            "format": "NHWC",
            "is_const": true,
            "const_value": [49, 49],
            "name": "size",
            "shape": [
              2
            ],
            "type": "int32",
    	"test": [7, 7.0]
          }
        ],
        "output_desc": [
          {
            "format": "NHWC",
            "name": "y",
            "shape": [
              4,
              48,
              48,
              16
            ],
            "type": "float"
          }
        ],
    	"attr": [
          {
            "name": "align_corners",
            "type": "bool",
            "value": false
          },
          {
            "name": "half_pixel_centers",
            "type": "bool",
            "value": false
          }
        ]
      }
    ]

    上述单算子转换后的离线模型为:0_ResizeBilinearV2_1_1_4_16_16_16_3_1_2_0_1_4_48_48_16.om

  • 可选输入(optional input):
    [
      {
        "op": "MatMulV2",
        "input_desc": [
          {
            "format": "ND",
            "shape": [16, 16],
            "type": "float"
          },
          {
            "format": "ND",
            "shape": [16, 16],
            "type": "float"
          },
          {
            "format": "RESERVED",
            "shape": [16, 16],
            "type": "UNDEFINED"
          },
          {
            "format": "RESERVED",
            "shape": [16, 16],
            "type": "UNDEFINED"
          }
        ],
        "attr": [
        {  
            "name": "transpose_x1",
            "type": "bool",
            "value": false
        },
        {
            "name": "transpose_x2",
            "type": "bool",
            "value": false
        }
        ],
        "output_desc": [
          {
            "format": "ND",
            "shape": [16, 16],
            "type": "float"
          }
    	]
      }
    ]
    

    上述单算子转换后的离线模型为:0_MatMulV2_0_2_16_16_0_2_16_16_26_40_16_16_26_40_16_16_0_2_16_16.om

  • 输入个数不确定(动态输入场景

    该场景下,单算子的输入个数不确定。比如AddN单算子:

    • 构造的单算子json文件使用动态输入dynamic_input参数,而不使用Tensor的名称name参数,该场景下构造的描述文件为:

      该场景下算子的dynamic_input取值必须和算子信息库中该算子定义的输入name的取值相同。

      具体设置几个输入,由AddN单算子描述文件属性参数中N的取值决定,用户可以自行修改输入的个数,但是必须和属性中N的取值匹配。(该说明仅针对AddN算子生效,其他动态输入算子的约束以具体算子为准。)
      [
          {
              "op": "AddN",
              "input_desc": [
                  {
                      "dynamic_input": "x",
                      "format": "NCHW",
                      "shape": [1,3,166,166],
                      "type": "float32"
                  },
                  {
                      "dynamic_input": "x",
                      "format": "NCHW",
                      "shape": [1,3,166,166],
                      "type": "int32"
                  },
                  {
                      "dynamic_input": "x",
                      "format": "NCHW",
                      "shape": [1,3,166,166],
                      "type": "float32"
                  }
              ],
              "output_desc": [
                  {
                      "format": "NCHW",
                      "shape": [1,3,166,166],
                      "type": "float32"
                  }
              ],
              "attr": [
                  {
                      "name": "N",
                      "type": "int",
                      "value": 3
                  }
              ]
          }
      ]
    • 构造的单算子json文件使用Tensor的名称name参数,而不使用动态输入dynamic_input参数,该场景下构造的描述文件为:

      该场景下算子的name取值必须和算子原型定义中算子的输入名称相同,根据输入的个数自动生成x0、x1、x2……。

      具体设置几个Tensor名称,由AddN单算子描述文件属性参数中N的取值决定,用户可以自行修改Tensor名称的个数,但是必须和属性中N的取值匹配,例如N取值为3,则name取值分别设置为x0、x1、x2。(该说明仅针对AddN算子生效,其他动态输入算子的约束以具体算子为准。)
      [
          {
              "op": "AddN",
              "input_desc": [
                  {
      		"name":"x0",
                      "format": "NCHW",
                      "shape": [1,3,166,166],
                      "type": "float32"
                  },
                  {
      		"name":"x1",
                      "format": "NCHW",
                      "shape": [1,3,166,166],
                      "type": "int32"
                  },
                  {
      		"name":"x2"
                      "format": "NCHW",
                      "shape": [1,3,166,166],
                      "type": "float32",
      
                  }
              ],
              "output_desc": [
                  {
                      "format": "NCHW",
                      "shape": [1,3,166,166],
                      "type": "float32"
                  }
              ],
              "attr": [
                  {
                      "name": "N",
                      "type": "int",
                      "value": 3
                  }
              ]
          }
      ]

    上述单算子转换后的离线模型为:0_AddN_0_0_1_3_166_166_3_0_1_3_166_166_0_0_1_3_166_166_0_0_1_3_166_166.om

  • 模糊编译场景

    如果用户无法获取算子的shape范围,又想编译一次达到多次执行推理的目的时,可以使用该特性。该场景下构造的单算子描述文件如下,当前仅支持Transformer网络模型涉及的算子:

    [   
        {
         "compile_flag":1
          "op": "Add",
          "input_desc": [
            {
              "format": "ND",
              "shape": [3,3],
              "type": "int32"
            },
            {
              "format": "ND",
              "shape": [3,3],
              "type": "int32"
            }
          ],
          "output_desc": [
            {
              "format": "ND",
              "shape": [3,3],
              "type": "int32"
            }
          ]
        }
      ]