Documentation

Operator Prototype Definition

This section describes the concept and obtaining method of the operator prototype.

Overview

The operator prototype defines the constraints for running an operator on Ascend AI Processor. It mainly reflects the mathematical meaning of the operator, including its inputs, outputs, and attributes.

In the operator development phase, operator registration starts with the REG_OP macro, and is followed by the INPUT, OUTPUT, and ATTR calls, which are connected using periods (.). The registration ends with OP_END_FACTORY_REG. A code sample is as follows.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
 
namespace ge{
    REG_OP(OpType) // Operator type name
    .INPUT(x1, TensorType({DT_FLOAT, DT_INT32}))
    .INPUT(x2, TensorType({DT_FLOAT, DT_INT32})) 
    .DYNAMIC_INPUT(a, TensorType({DT_FLOAT, DT_INT32}))
    .OPTIONAL_INPUT(b, TensorType({DT_FLOAT}))        
    .OUTPUT(y, TensorType({DT_FLOAT, DT_INT32})) 
    .DYNAMIC_OUTPUT(c, TensorType({DT_FLOAT, DT_INT32}))
    .ATTR(x, Type, DefaultValue)
    .REQUIRED_ATTR(x, Type)
    .OP_END_FACTORY_REG(OpType)
}
  • REG_OP(OpType)

    OpType: operator type registered with the custom OPP of Ascend AI Processor.

  • INPUT(x1, TensorType({DT_FLOAT, DT_UINT8, ...}))
    Registers the input information of an operator.
    • x1: a macro parameter for the input name.
    • TensorType({DT_FLOAT, DT_UINT8, ...}): a list of supported data types for the input.

    If the operator has multiple inputs, register each input with a separate statement INPUT(x, TensorType({DT_FLOAT, DT_UINT8, ...})).

  • DYNAMIC_INPUT(a, TensorType({DT_FLOAT, DT_INT32, ...}))
    Registers the input information for a dynamic multi-input operator.
    • a: a macro parameter for the input name. It is automatically generated and numbered in ascending order during graph execution according to the number of inputs, for example, a0, a1, a2, and more.
    • TensorType({DT_FLOAT, DT_UINT8, ...}): a list of supported data types for the input.
  • OPTIONAL_INPUT(b, TensorType({DT_FLOAT, ...}))
    Registers an optional operator input.
    • b: a macro parameter for the input name.
    • TensorType({DT_FLOAT, ...}): a list of supported data types for the input.
  • OUTPUT(y, TensorType({DT_FLOAT, DT_UINT8, ...}))
    Registers the output information of an operator.
    • y: a macro parameter for the output name.
    • TensorType({DT_FLOAT, DT_UINT8, ...}): a list of supported data types for the output.

    If the operator has multiple outputs, register each output with a separate statement OUTPUT(y, TensorType({DT_FLOAT, DT_UINT8, ...})).

  • DYNAMIC_OUTPUT(c, TensorType({DT_FLOAT, DT_INT32}))
    Registers the output information for a dynamic multi-output operator.
    • c: a macro parameter for the output name. It is automatically generated and numbered in ascending order during graph execution according to the number of outputs, for example, c0, c1, c2, and more.
    • TensorType({DT_FLOAT, DT_UINT8, ...}): a list of supported data types for the output.
  • ATTR(x, Type, DefaultValue)

    Registers an operator attribute, including attribute name, type, and default value. The default value is used if the attribute value is not specified.

    For example, ATTR(mode, Int, 1) registers a mode attribute of type int64_t with default value 1.

    If the operator has multiple attributes, register each with a separate statement ATTR(x, Type, DefaultValue) or REQUIRED_ATTR(d, Type).

  • REQUIRED_ATTR(d, Type)

    Registers an operator attribute, including attribute name and type. No default value is predefined, indicating that the attribute value must be specified during development.

    If the operator has multiple attributes, register each with a separate statement ATTR(x, Type, DefaultValue) or REQUIRED_ATTR(d, Type).

  • OP_END_FACTORY_REG(OpType): ends operator registration. The OpType argument must be consistent with that of REG_OP(OpType).

For details about the mapping between data types such as DT_FLOAT and DT_UINT8, see DataType. The DT_BF16 data type in this file is supported only by the following product models:

Obtaining Operator Prototype

During the model construction phase, it is necessary to understand the operator prototype basics (including inputs, outputs, and attributes) in order to create operator instances and construct your own graph.

  • For a custom operator, obtain its prototype definition header file to understand its prototype definition.
    Refer to the header file of the operator prototype definition in {Ascend-CANN-Toolkit installation directory}/ascend-toolkit/latest/opp/vendors/<vendor_name>/op_proto/inc. The following provides an example. 
    1
2
3
4
5
6
    		 
    REG_OP(SubMConv3dCube) 
    .INPUT(x, TensorType({DT_FLOAT16})) 
    .INPUT(filter, TensorType({DT_FLOAT16}))
    .OUTPUT(y, TensorType({DT_FLOAT16}))
    .ATTR(is_first, Bool, false)
    .OP_END_FACTORY_REG(SubMConv3dCube)
  • For a built-in operator, obtain the operator prototype using either of the following methods:
    • Obtain the operator prototype from CANN Operator Specifications, as shown in Figure 1.
      Figure 1 Viewing the operator prototype information
    • Refer to the header file of the operator prototype definition in {Ascend-CANN-Toolkit installation directory}/ascend-toolkit/latest/opp/built-in/op_proto/inc. The following provides an example. 
      1
2
3
4
5
6
      		 
      REG_OP(SoftmaxV2)
    .INPUT(x, TensorType({DT_DOUBLE, DT_FLOAT16, DT_BF16, DT_FLOAT}))
    .OUTPUT(y, TensorType({DT_DOUBLE, DT_FLOAT16, DT_BF16, DT_FLOAT}))
    .ATTR(axes, ListInt, {-1})
    .ATTR(half_to_float, Bool, false)
    .OP_END_FACTORY_REG(SoftmaxV2)
Parent topic: Preparations