What Is a Single-Operator Description File?

compile_flag

INT32

Deprecated.

Build type. The values are as follows:

• 0: precise build. In precise build mode, the system performs build based on the user-defined shape without any escape. However, this flag is invalid for AI CPU operators.
• 1: fuzzy build. In fuzzy build mode, the system generalizes the runtime dimensions of dynamic-shape operators before build. Use this feature if you want to run multiple inferences based on one build while the operator's shape range is unknown.

The default value is 0.

Only operators of the Transformer network are supported currently.

No

op

string

Operator type.

Yes

name

string

Name of a single-operator model file.

If name is not set, the default naming rule of the model file is SN_Operator type_Input description (dataType_format_shape)_Output description (dataType_format_shape), for example, 0_Add_3_2_3_3_3_2_3_3_3_2_3_3.om.

You can check the enum values of dataType and format from the ${INSTALL_DIR}/include/graph/types.h file. The enum values are sorted in ascending order. Replace ${INSTALL_DIR} with the CANN component directory. For example, if the installation is performed by the root user, the default file storage path is /usr/local/Ascend/cann.

No

input_desc

TensorDesc array

Operator input description.

Yes

output_desc

TensorDesc array

Operator output description.

Yes

attr

Attr array

Operator attribute.

No

dynamic_input

string

(Optional) Dynamic input. The value must be the same as the input name field defined in the operator information library.

Assume that an operator has two groups of dynamic inputs, which are defined as follows:

.DYNAMIC_INPUT(x,...)

.DYNAMIC_INPUT(y,...)

It indicates that two groups of dynamic inputs are defined (x and y). The number of inputs in each group is determined by the number of dynamic_input fields. For details, see the description of the name parameter in the TensorDesc array.

• If the constructed single-operator description file contains the name field, this parameter is optional.
• If the constructed single-operator description file does not contain the name field, this parameter is required.
• If both the dynamic_input and name parameters exist, the name parameter is used.

format

string

(Mandatory) Format used during tensor compute, also called runtime format, which is the format used for compute on the device.

The mapping between the possible formats and enum values is as follows:

• NCHW: 0
• NHWC: 1
• ND: 2, any format.
• NC1HWC0: 3, a 5D format.
• FRACTAL_Z: 4, a format to define convolution weights.
• FRACTAL_NZ: 29, a fractal format. For details about the preceding formats, see Basic Concepts.
• RESERVED: 40. For any optional input without data, set format to RESERVED and set type to UNDEFINED; for any optional input with data, set format and type to the actual format and type of the data.

After your model is converted, you will see the supported format represented as the corresponding enum value in the file name of the generated offline model (.om) file. For example, 1 maps to format NHWC.

origin_format

string

(Optional) Original format of the tensor input, that is, the original image format without any conversion.

If this field is not included, the format used during tensor computation is the same as the original format by default.

name

string

(Optional) Tensor name. It is required when the operator has dynamic inputs.

This parameter sets the input names in each group of dynamic inputs. Each input name consists of the input name defined in the operator prototype plus an ID. The ID is determined based on the number of dynamic inputs and indexed from 0.

• If the constructed single-operator description file contains the dynamic_input field, this parameter is optional.
• If the constructed single-operator description file does not contain the dynamic_input field, this parameter is required.
• If both the dynamic_input and name parameters exist, the name parameter is used.

shape

Array of ints

(Mandatory) Shape used during tensor compute, for example, [1, 224, 224, 3]. The actual shape product cannot be greater than the maximum int32 value (2147483647).

• For a static shape:

  The dimensions and shape are fixed. In this scenario, shape_range does not need to be configured.

• For a constant shape:

  To specify the input and output shapes of an operator as scalars, set this parameter to "[]" format, for example, "shape": []. In this scenario, shape_range does not need to be configured.

• For a dynamic shape, the shape value can be like the following:
  • If the shape is determined, its unknown dimensions are set to -1, for example, [16, -1, 20, -1]. In this scenario, this parameter must be used together with shape_range, which specifies the value range for dimensions setting to -1. The sample code is as follows:

            "shape": [-1,16],
            "shape_range": [[0,32]],

  • If the shape is unknown, its value is -2, for example, "shape": [-2]. In this scenario, shape_range is not supported in the current version and does not need to be configured.

  In the dynamic shape operator execution scenario, the version of the operator library package in the operator execution environment must be the same as that in the operator model compilation environment. Otherwise, an error is reported when the operator is loaded. For details, see Failure in Loading a Single-Operator Due to an OPP Version Issue.

origin_shape

string

(Optional) Original shape of the input tensor.

If this parameter is not included, the shape used during tensor computation is the same as the original shape by default.

type

string

(Mandatory) Tensor data type. The mapping between the possible data types and enum values is as follows:

• bool: 12
• int8: 2
• uint8: 4
• int16: 6
• uint16: 7
• int32: 3
• uint32: 8
• int64: 9
• uint64: 10
• float16/fp16/half: 1
• float/float32: 0
• double: 11
• complex32: 33
• complex64: 16
• complex128: 17
• uint1: 30
• bfloat16: 27
• int4: 29
• UNDEFINED: 28. For any optional input without data, set type to UNDEFINED and format to RESERVED; for any optional input with data, set format and type to the actual format and type of the data.

After your model is converted, you will see the supported data types represented as the corresponding enum values in the file name of the generated offline model (.om) file. For example, 2 maps to type int8.

shape_range

Array of int[2]s

(Optional) Value range of an unknown/dynamic shape (excluding the scenario where shape is -2).

For example, for shape [16, -1, 20, -1], -1 indicates an unknown dimension.

Assume that shape_range is [1, 128], [1, -1], where [1, 128] indicates the value range from 1 to 128, corresponding to the first -1 dimension in the shape value, and [1, -1] indicates the value range from 1 to infinity, corresponding to the second -1 dimension accordingly.

is_const

bool

(Optional) Whether the input is a constant.

• true: constant
• false (default): not constant

const_value

list

(Optional) Constant value.

Currently, this parameter supports only a one-dimensional list. The number of the list items to configure is determined by the value of shape. For example, if the value of shape is 2, there are two list items in const_value.

The data type of const_value is determined by type. If type is set to float16, single-operator build automatically converts the value of const_value to the float16 format.

name

string

(Mandatory) Attribute name.

type

string

(Mandatory) Attribute value type. The options are as follows:

• bool
• int
• float
• string
• list_bool
• list_int
• list_float
• list_string
• list_list_int
• data_type

value

Determined by the value of type

(Mandatory) Attribute value, which varies with type. The value examples are as follows:

• bool: true/false
• int: 10
• float: 1.0
• string: "NCHW"
• list_bool: [false, true]
• list_int: [1, 224, 224, 3]
• list_float: [1.0, 0.0]
• list_string: ["str1","str2"]
• list_list_int: [[1, 3, 5, 7], [2, 4, 6, 8]]
• data_type: "DT_FLOAT" or the number corresponding to the enum value, for example, 0.

  For details about other values, see the enum values of DataType or the numbers corresponding to the enum values in ${INSTALL_DIR}/include/graph/types.h. Replace ${INSTALL_DIR} with the CANN component directory. For example, if the installation is performed by the root user, the default file storage path is /usr/local/Ascend/cann.