Operator Prototype Definition

Overview

The operator prototype describes the input, output, attributes, and implementation information of an operator on the Ascend AI Processor.

In the operator development phase, operator registration starts with the REG_OP macro, is followed by the INPUT, OUTPUT, and ATTR calls, which are connected using periods (.), and ends with OP_END_FACTORY_REG. A code sample is as follows.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12

namespace ge{
    REG_OP(OpType) // Operator type name
    .INPUT(x1, TensorType({DT_FLOAT, DT_INT32}))
    .INPUT(x2, TensorType({DT_FLOAT, DT_INT32})) 
    .DYNAMIC_INPUT(a, TensorType({DT_FLOAT, DT_INT32}))
    .OPTIONAL_INPUT(b, TensorType({DT_FLOAT}))        
    .OUTPUT(y, TensorType({DT_FLOAT, DT_INT32})) 
    .DYNAMIC_OUTPUT(c, TensorType({DT_FLOAT, DT_INT32}))
    .ATTR(x, Type, DefaultValue)
    .REQUIRED_ATTR(x, Type)
    .OP_END_FACTORY_REG(OpType)
}

• REG_OP(OpType)

  OpType: operator type registered with the custom operator library of Ascend AI Processor.

• INPUT(x1, TensorType({DT_FLOAT,DT_UINT8,... }))
  Registers the input information of an operator.

  • x1: a macro parameter for the input name.
  • TensorType({DT_FLOAT,DT_UINT8,... }): a list of supported data types for the input.

  If the operator has multiple inputs, register each input with a separate statement INPUT(x, TensorType({DT_FLOAT,DT_UINT8,... })).

• DYNAMIC_INPUT(a, TensorType({DT_FLOAT, DT_INT32,... }))
  Registers the input information for a dynamic multi-input operator.

  • a: a macro parameter for the input name. It is automatically generated and numbered in ascending order during graph execution according to the number of inputs, for example, a0, a1, and a2.
  • TensorType({DT_FLOAT, DT_INT32,... }): a list of supported data types for the input.
• OPTIONAL_INPUT(b, TensorType({DT_FLOAT,... }))
  Registers an optional operator input.

  • b: a macro parameter for the input name.
  • TensorType({DT_FLOAT,... }): a list of supported data types for the input.
• OUTPUT(y, TensorType({DT_FLOAT,DT_UINT8,... }))
  Registers the output information of an operator.

  • y: a macro parameter for the output name.
  • TensorType({DT_FLOAT,DT_UINT8,... }): a list of supported data types for the output.

  If the operator has multiple outputs, register each output with a separate statement OUTPUT(y, TensorType({ DT_FLOAT,DT_UINT8,... })).

• DYNAMIC_OUTPUT(c, TensorType({DT_FLOAT, DT_INT32,... }))
  Registers the output information for a dynamic multi-output operator.

  • c: a macro parameter for the output name. It is automatically generated and numbered in ascending order during graph execution according to the number of outputs, for example, c0, c1, and c2.
  • TensorType({DT_FLOAT, DT_INT32,... }): a list of supported data types for the output.
• ATTR(x, Type,DefaultValue)

  Registers an operator attribute, including attribute name, type, and default value. The system will use the default value if no attribute value is set.

  For example, ATTR(mode, Int, 1) registers a mode attribute of type int64_t with default value 1.

  If the operator has multiple attributes, each attribute needs to be registered using one ATTR(x, Type, DefaultValue) statement.

• REQUIRED_ATTR(d, Type)

  Registers an operator attribute, including attribute name and type. No default value is predefined, indicating that the attribute value must be specified during development.

  If the operator has multiple attributes, each attribute needs to be registered using one REQUIRED_ATTR(d, Type) statement.

• OP_END_FACTORY_REG(OpType): ends operator registration. The OpType argument must be consistent with that of REG_OP(OpType).

For details about the mapping between data types such as DT_FLOAT and DT_UINT8, see DataType. Only the following products support the DT_BF16 data type in the file.

Atlas A2 training products / Atlas A2 inference products

Atlas A3 training products / Atlas A3 inference products

Obtaining Operator Prototype

During the model construction phase, it is necessary to understand the operator prototype basics (including inputs, outputs, and attributes) in order to create operator instances and construct your own graph.

• For a custom operator, obtain its prototype definition header file to understand its prototype definition.
  Obtain the operator prototype definition from the header file in ${INSTALL_DIR}/opp/vendors/<vendor_name>/op_proto/inc.

  1 2 3 4 5 6

  REG_OP(SubMConv3dCube) .INPUT(x, TensorType({DT_FLOAT16})) .INPUT(filter, TensorType({DT_FLOAT16})) .OUTPUT(y, TensorType({DT_FLOAT16})) .ATTR(is_first, Bool, false) .OP_END_FACTORY_REG(SubMConv3dCube)

  Replace ${INSTALL_DIR} with the CANN component directory. For example, if the installation is performed by the root user, the default file storage path is /usr/local/Ascend/cann.

• For a built-in operator, obtain the operator prototype using either of the following methods:
  • Obtain the operator prototype from CANN Operator Specifications, as shown in Figure 1.
    Figure 1 Viewing the operator prototype information
    
  • Obtain the operator prototype from the prototype definition header file ${INSTALL_DIR}/opp/built-in/op_graph/inc. For example:

    1 2 3 4 5 6

    REG_OP(SoftmaxV2) .INPUT(x, TensorType({DT_DOUBLE, DT_FLOAT16, DT_BF16, DT_FLOAT})) .OUTPUT(y, TensorType({DT_DOUBLE, DT_FLOAT16, DT_BF16, DT_FLOAT})) .ATTR(axes, ListInt, {-1}) .ATTR(half_to_float, Bool, false) .OP_END_FACTORY_REG(SoftmaxV2)

    Replace ${INSTALL_DIR} with the CANN component directory. For example, if the installation is performed by the root user, the default file storage path is /usr/local/Ascend/cann.