Documentation

Operator Analysis

Overview

This section uses the TBE operator or AI CPU operator "Add" as an example to show the workflow of operator analysis.

TBE Operator Analysis

Before developing an Add operator by using the TBE DSL, you need to determine the operator functionality, inputs, outputs, select an operator development mode, and name the operator type and implementation function.

  1. Specify the operator function and mathematical expression.

    Specify the mathematical expression of the Add operator as follows:

     z=x+y

    The Add operator adds two inputs and returns a result.

  2. Specify the inputs and output.
    • The Add operator has two inputs, x and y, and outputs the result z.
    • The supported input data types include float16, float32, and int32. The output has the same data type as the inputs.
    • The operator inputs support all shapes. The output has the same shape as the inputs.
    • The operator input supports the following formats: NCHW, NC1HWC0, NHWC, and ND.
  3. Determine the operator development mode and the compute API.
    1. The compute process involves only the addition operation. The tbe.dsl.vadd(lhs, rhs) API can be used to implement "x + y" for preliminary analysis. For details, see "Operator Development > Operator Code Implementation (TBE DSL)" in the TBE & AI CPU Operator Development Guide.
    2. The tbe.dsl.vadd(lhs, rhs) API requires that the two input tensors have the same shape. Therefore, you need to obtain the larger shape of the two input tensors, and then call the tbe.dsl.broadcast(var, shape, output_dtype=None) API to broadcast the input tensors to the specified shape.
  4. Specify the operator implementation file name, operator implementation function name, and OpType.

    The naming rules are as follows:

    • Name OpType in upper camel case and indicate the separation of words with a single capitalized letter.
    • Name the operator file and operator function in either of the following ways:
      • To create user-defined names, configure opFile.value and opInterface.value in Operator Information Library Definition.
      • If opFile.value and opInterface.value in the Operator Information Library Definition are not configured, FE obtains the operator file name and function name by replacing the OpType as follows:
        • Replace the first uppercase letter with a lowercase letter.

          Example: Abc -> abc

        • Replace each uppercase letter following lowercase letters with an underscore (_) and a lowercase letter.

          Example: AbcDef -> abc_def

        • Uppercase letters following a digit or an uppercase letter are regarded as a character string. If there is a lowercase letter after this string, replace the last uppercase letter in this string with an underscore (_) and a lowercase letter, and replace the other uppercase letters with lowercase letters. If there is no lowercase letter after the string, directly replace the string with lowercase letters.

          Examples: ABCDef -> abc_def; Abc2DEf -> abc2d_ef; Abc2DEF -> abc2def; ABC2dEF -> abc2d_ef

    In this example, OpType of the operator is defined as Add. Uncapitalize the first letter to obtain the operator implementation file name and implementation function name, that is, add.

  5. Based on the preceding analysis, the design specifications of the Add operator are as follows:
    Table 1 Design specifications

    OpType

    Add

    Operator Input

    name: x1

    shape: all

    data type:

    float16, float32, int32

    format:

    NCHW,NC1HWC0,NHWC,ND

    name: x2

    shape: all

    data type:

    float16, float32, int32

    format:

    NCHW,NC1HWC0,NHWC,ND

    Operator Output

    name: y

    shape: all

    data type:

    float16, float32, int32

    format:

    NCHW,NC1HWC0,NHWC,ND

    Main DSL API for Operator Implementation

    tbe.dsl.broadcast(var, shape, output_dtype=None)

    tbe.dsl.vadd(lhs, rhs)

    Operator Implementation File/Function Name

    add

AI CPU Operator Analysis

Before developing an AI CPU operator, determine the operator functionality, inputs, outputs, type, implementation function name, and more.

  1. Specify the operator function and mathematical expression.

    Take the Add operator as an example. The mathematical expression of the Add operator is as follows:

     z=x+y

    The Add operator adds two inputs and returns a result.

  2. Specify the inputs and output.
    • The Add operator has two inputs, x and y, and outputs the result z.
    • The supported input data types include float16, float32, and int32. The output has the same data type as the inputs.
    • The operator inputs support all shapes. The output has the same shape as the inputs.
    • The operator inputs support the following formats: NCHW, NHWC, and ND.
  3. Specify the operator implementation file name and OpType.
    • Name OpType in upper camel case and indicate the separation of words with a single capitalized letter.
    • Name the operator file in either of the following ways:

      Name the operator implementation file after OpType as follows:

      The rules are as follows:
      • Replace the first uppercase letter with a lowercase letter.

        Example: Abc -> abc

      • Replace each uppercase letter following lowercase letters with an underscore (_) and a lowercase letter.

        Example: AbcDef -> abc_def

      • Uppercase letters following a digit or an uppercase letter are regarded as a character string. If there is a lowercase letter after this string, replace the last uppercase letter in this string with an underscore (_) and a lowercase letter, and replace the other uppercase letters with lowercase letters. If there is no lowercase letter after the string, directly replace the string with lowercase letters.

        Examples: ABCDef -> abc_def; Abc2DEf -> abc2d_ef; Abc2DEF -> abc2def; ABC2dEF -> abc2d_ef

    In this example, OpType of the operator is defined as Add. You are advised to name the deliverables as follows:
    • Code implementation (or kernel implementation) files of the operator: add_kernel.h and add_kernel.cc
    • Plugin implementation file: add_kernel_plugin.cc
    • Prototype definition files: add.h and add.cc
    • Information definition file: add.ini
  4. Based on the preceding analysis, the design specifications of the Add operator are as follows:
    Table 2 Design specifications

    OpType

    Add

    Operator Input

    name: x

    shape: all

    data type:

    float16, float32, int32

    name: y

    shape: all

    data type:

    float16, float32, int32

    Operator Output

    name: z

    shape: all

    data type:

    float16, float32, int32

    Operator Implementation File Name

    add
Parent topic: Operator Development