What Is a Single-Operator Description File?

compile_flag

INT32

Deprecated.

Build type. The options are as follows:

• 0: precise build. In this mode, the system uses the user-specified shape directly for build without any internal dimension transformation. This flag does not apply to AI CPU operators.
• 1: fuzzy build. In this mode, the system generalizes the variable dimensions of dynamic-shape operators before build. Use this feature if you want to run multiple inferences based on one build while the operator's shape range is unknown.

The default value is 0.

Only operators of the Transformer network are supported currently.

No

op

String

Operator type.

Yes

name

String

Name of a single-operator model file.

If name is not set, the default naming rule of the model file is SN_Operator type_Input descriptions (dataType_format_shape)_Output descriptions (dataType_format_shape), for example, 0_Add_3_2_3_3_3_2_3_3_3_2_3_3.om.

You can check the enum values of dataType and format from the ${INSTALL_DIR}/include/graph/types.h file. The enum values are sorted in ascending order.

Replace ${INSTALL_DIR} with the CANN component directory. For example, if the installation is performed by the root user, the default file storage path is /usr/local/Ascend/cann.

No

input_desc

TensorDesc array

Operator input descriptions.

Yes

output_desc

TensorDesc array

Operator output descriptions.

Yes

attr

Attr array

Operator attributes.

No

dynamic_input

String

(Optional) Dynamic input. The value must be the same as the input name field defined in the operator information library.

This parameter sets the grouping and quantity of dynamic inputs for operators. For example, an operator has the following dynamic inputs in its operator prototype definition:

.DYNAMIC_INPUT(x,...)

.DYNAMIC_INPUT(y,...)

It indicates two groups of dynamic inputs, namely x and y. The number of inputs in each group is determined by the number of dynamic_input fields. For details, see the description of the name parameter in the TensorDesc array.

• If the constructed single-operator description file contains the name field, this parameter is optional.
• If the constructed single-operator description file does not contain the name field, this parameter is mandatory.
• If both dynamic_input and name parameters exist, the name parameter takes precedence.

format

String

(Mandatory) Format used during tensor compute, also called runtime format, which is the format used for compute on the device.

The mapping between the possible formats and enum values is as follows:

• NCHW: 0.
• NHWC: 1.
• ND: 2, any format.
• NC1HWC0: 3, a 5D format.
• FRACTAL_Z: 4, a format to define convolution weights.
• FRACTAL_NZ: 29, a fractal format. For details about the preceding formats, see Basic Concepts.
• RESERVED: 40. For any optional input without data, set format to RESERVED and set type to UNDEFINED; for any optional input with data, set format and type to the actual format and type of the data.

After model conversion, you will see the supported format represented as the corresponding enum value in the name of the generated offline model (.om) file. For example, 1 maps to format NHWC.

origin_format

String

(Optional) Original format of the tensor input, that is, the original image format without any conversion.

If this field is not included, the format used during tensor computation is the same as the original format by default.

name

String

(Optional) Tensor name. It is required when the operator has dynamic inputs.

This parameter sets the input names in each group of dynamic inputs. Each input name consists of the input name defined in the operator prototype plus an ID. The ID is determined based on the number of dynamic inputs and indexed from 0.

• If the dynamic_input parameter has been configured in the constructed single-operator description file, this parameter is optional.
• If the constructed single-operator description file does not contain the dynamic_input field, this parameter is mandatory.
• If both dynamic_input and name parameters exist, the name parameter takes precedence.

shape

Array of ints

(Mandatory) Shape used during tensor computation, for example, [1, 224, 224, 3]. The actual shape product cannot be greater than the maximum int32 value (2147483647).

• For a static shape:

  The dimensions and shape are fixed. In this scenario, shape_range does not need to be configured.

• For a constant shape:

  To specify the input and output shapes of an operator as scalars, set this parameter to the "[]" format, for example, "shape": []. In this scenario, shape_range does not need to be configured.

• For a dynamic shape:
  • If the number of shape dimensions is determined, its unknown dimensions are set to -1, for example, [16, -1, 20, -1]. In this scenario, this parameter must be used together with shape_range, which specifies the value range for dimensions set to -1. For example:

    "shape": [-1,16],

    "shape_range": [[0,32]],

  • If the number of shape dimensions is unknown, set shape to -2, for example, "shape": [-2]. In this scenario, shape_range does not need to be configured (not supported in the current version).

  For dynamic-shape operator execution scenarios, the installed version of the operator library package in the runtime environment must match the version used in the model compilation environment. Otherwise, an error occurs during operator loading. For details, see Failure in Loading a Single Operator Due to an OPP Version Issue.

origin_shape

String

(Optional) Original shape of the input tensor.

If this parameter is not included, the shape used during tensor computation is the same as the original shape by default.

type

String

(Mandatory) Data type of the tensor. The mapping between the possible data types and enum values is as follows:

• bool: 12
• int8: 2
• uint8: 4
• int16: 6
• uint16: 7
• int32: 3
• uint32: 8
• int64: 9
• uint64: 10
• float16/fp16/half: 1
• float/float32: 0
• double: 11
• complex32: 33
• complex64: 16
• complex128: 17
• uint1: 30
• bfloat16: 27
• int4: 29
• UNDEFINED: 28. For any optional input without data, set type to UNDEFINED and format to RESERVED; for any optional input with data, set format and type to the actual format and type of the data.

After model conversion, you will see the supported data types represented as the corresponding enum values in the name of the generated offline model (.om) file. For example, 2 maps to type int8.

shape_range

Array of int[2]s

(Optional) Value ranges of unknown dimensions in dynamic shape scenarios (excluding the scenario where shape is -2).

For example, for shape [16, -1, 20, -1], -1 indicates an unknown dimension.

Assume that shape_range is [1, 128], [1, -1], where [1, 128] indicates the value range from 1 to 128, corresponding to the first -1 dimension in the shape value, and [1, -1] indicates the value range from 1 to infinity, corresponding to the second -1 dimension accordingly.

is_const

Bool

(Optional) Whether the input is a constant.

• true: constant
• false (default): not constant

const_value

List

(Optional) Constant values.

Currently, this parameter supports only a one-dimensional list. The number of the list items to configure is determined by the value of shape. For example, if the value of shape is 2, there are two list items in const_value.

Its data type is determined by type. If type is set to float16, single-operator build automatically converts the value of const_value to the float16 format.

name

String

(Mandatory) Attribute name.

type

String

(Mandatory) Type of the attribute value. The following types are supported:

• bool
• int
• float
• string
• list_bool
• list_int
• list_float
• list_string
• list_list_int
• data_type

value

Determined by the value of type

(Mandatory) Attribute value, which varies with type. The value examples are as follows:

• bool: true/false
• int: 10
• float: 1.0
• string: "NCHW"
• list_bool: [false, true]
• list_int: [1, 224, 224, 3]
• list_float: [1.0, 0.0]
• list_string: ["str1","str2"]
• list_list_int: [[1, 3, 5, 7], [2, 4, 6, 8]]
• data_type: "DT_FLOAT" or the number corresponding to the enum value, for example, 0.

  For details about other values, see the enum values of DataType or the numbers corresponding to the enum values in ${INSTALL_DIR}/include/graph/types.h. (Replace ${INSTALL_DIR} with the CANN component directory. For example, if the installation is performed by the root user, the default file storage path is /usr/local/Ascend/cann.)