本章节中的单算子是基于Ascend IR定义的,描述文件为json格式。关于单算子的定义请参见《算子清单》。
单算子生成的离线模型文件命名规则为:序号_opType_输入的描述(dataType_format_shape)_输出的描述(dataType_format_shape)。
dataType以及format对应枚举值请从${INSTALL_DIR}/include/graph/types.h文件中查看,枚举值从0开始依次递增。其中,${INSTALL_DIR}请替换为CANN软件安装后文件存储路径。例如,若安装的Ascend-cann-toolkit软件包,则安装后文件存储路径为:$HOME/Ascend/ascend-toolkit/latest。
[ { "op": "GEMM", "input_desc": [ { "format": "ND", "shape": [16, 16], "type": "float16" }, { "format": "ND", "shape": [16, 16], "type": "float16" }, { "format": "ND", "shape": [16, 16], "type": "float16" }, { "format": "ND", "shape": [], "type": "float16" }, { "format": "ND", "shape": [], "type": "float16" } ], "output_desc": [ { "format": "ND", "shape": [16, 16], "type": "float16" } ], "attr": [ { "name": "transpose_a", "type": "bool", "value": false }, { "name": "transpose_b", "type": "bool", "value": false } ] } ]
上述单算子转换后的离线模型为:0_GEMM_1_2_16_16_1_2_16_16_1_2_16_16_1_2_1_2_1_2_16_16.om
[ { "op": "Conv2D", "input_desc": [ { "format": "NCHW", "shape": [1, 3, 16, 16], "type": "float16" }, { "format": "NCHW", "shape": [3, 3, 3, 3], "type": "float16" } ], "output_desc": [ { "format": "NCHW", "shape": [1, 3, 16, 16], "type": "float16" } ], "attr": [ { "name": "strides", "type": "list_int", "value": [1, 1, 1, 1] }, { "name": "pads", "type": "list_int", "value": [1, 1, 1, 1] }, { "name": "dilations", "type": "list_int", "value": [1, 1, 1, 1] } ] } ]
上述单算子转换后的离线模型为:0_Conv2D_1_0_1_3_16_16_1_0_3_3_3_3_1_0_1_3_16_16.om
[ { "op": "Add", "input_desc": [ { "format": "NC1HWC0", "origin_format": "NCHW", "shape": [8, 1, 16, 4, 16], "origin_shape": [8, 16, 16, 4], "type": "float16" }, { "format": "NC1HWC0", "origin_format": "NCHW", "shape": [8, 1, 16, 4, 16], "origin_shape": [8, 16, 16, 4], "type": "float16" } ], "output_desc": [ { "format": "NC1HWC0", "origin_format": "NCHW", "shape": [8, 1, 16, 4, 16], "origin_shape": [8, 16, 16, 4], "type": "float16" } ] } ]
上述单算子转换后的离线模型为:0_Add_1_3_8_1_16_4_16_1_3_8_1_16_4_16_1_3_8_1_16_4_16.om
该场景下,支持设置为常量的输入,新增is_const和const_value两个参数,分别表示是否为常量以及常量取值,const_value当前仅支持一维list配置,具体配置个数由shape取值决定,例如,如下样例中shape为2,则const_value中列表个数为2;const_value中取值类型由type决定,假设type取值为float16,则单算子编译时会自动将const_value中的取值转换为float16格式的取值。
[ { "op": "ResizeBilinearV2", "input_desc": [ { "format": "NHWC", "name": "x", "shape": [ 4, 16, 16, 16 ], "type": "float16" }, { "format": "NHWC", "is_const": true, "const_value": [49, 49], "name": "size", "shape": [ 2 ], "type": "int32", "test": [7, 7.0] } ], "output_desc": [ { "format": "NHWC", "name": "y", "shape": [ 4, 48, 48, 16 ], "type": "float" } ], "attr": [ { "name": "align_corners", "type": "bool", "value": false }, { "name": "half_pixel_centers", "type": "bool", "value": false } ] } ]
上述单算子转换后的离线模型为:0_ResizeBilinearV2_1_1_4_16_16_16_3_1_2_0_1_4_48_48_16.om
[ { "op": "MatMulV2", "input_desc": [ { "format": "ND", "shape": [16, 16], "type": "float" }, { "format": "ND", "shape": [16, 16], "type": "float" }, { "format": "RESERVED", "shape": [16, 16], "type": "UNDEFINED" }, { "format": "RESERVED", "shape": [16, 16], "type": "UNDEFINED" } ], "attr": [ { "name": "transpose_x1", "type": "bool", "value": false }, { "name": "transpose_x2", "type": "bool", "value": false } ], "output_desc": [ { "format": "ND", "shape": [16, 16], "type": "float" } ] } ]
上述单算子转换后的离线模型为:0_MatMulV2_0_2_16_16_0_2_16_16_26_40_16_16_26_40_16_16_0_2_16_16.om
该场景下,单算子的输入个数不确定。比如AddN单算子:
该场景下算子的dynamic_input取值必须和算子信息库中该算子定义的输入name的取值相同。
[ { "op": "AddN", "input_desc": [ { "dynamic_input": "x", "format": "NCHW", "shape": [1,3,166,166], "type": "float32" }, { "dynamic_input": "x", "format": "NCHW", "shape": [1,3,166,166], "type": "int32" }, { "dynamic_input": "x", "format": "NCHW", "shape": [1,3,166,166], "type": "float32" } ], "output_desc": [ { "format": "NCHW", "shape": [1,3,166,166], "type": "float32" } ], "attr": [ { "name": "N", "type": "int", "value": 3 } ] } ]
该场景下算子的name取值必须和算子原型定义中算子的输入名称相同,根据输入的个数自动生成x0、x1、x2……。
[ { "op": "AddN", "input_desc": [ { "name":"x0", "format": "NCHW", "shape": [1,3,166,166], "type": "float32" }, { "name":"x1", "format": "NCHW", "shape": [1,3,166,166], "type": "int32" }, { "name":"x2" "format": "NCHW", "shape": [1,3,166,166], "type": "float32", } ], "output_desc": [ { "format": "NCHW", "shape": [1,3,166,166], "type": "float32" } ], "attr": [ { "name": "N", "type": "int", "value": 3 } ] } ]
上述单算子转换后的离线模型为:0_AddN_0_0_1_3_166_166_3_0_1_3_166_166_0_0_1_3_166_166_0_0_1_3_166_166.om
如果用户无法获取算子的shape范围,又想编译一次达到多次执行推理的目的时,可以使用该特性。该场景下构造的单算子描述文件如下,当前仅支持Transformer网络模型涉及的算子:
[ { "compile_flag":1 "op": "Add", "input_desc": [ { "format": "ND", "shape": [3,3], "type": "int32" }, { "format": "ND", "shape": [3,3], "type": "int32" } ], "output_desc": [ { "format": "ND", "shape": [3,3], "type": "int32" } ] } ]