Documentation

Graph Construction from Operator Prototypes

This section describes how to use the operator prototype to build a complete graph step by step.

Overview

After operator prototypes are successfully registered by using the REG_OP macro, the corresponding derivative APIs are automatically generated (see Prototype Definition Derivative APIs. for more details). By using these derivative APIs, you can define operators in a graph, create a graph instance, and set the input and output operators in the graph, thereby completing the graph construction workflow.

Defining an Operator Using APIs Derived from Operator Prototypes

The following describes the general workflow of defining an operator using APIs derived from operator prototypes. For details about examples of defining operators (mainly data nodes and compute nodes), see Operator Expression.

  1. Include the header file. 
    1
    		 
    #include "all_ops.h"
    • For a built-in operator, include the all_ops.h header file in /opp/built-in/op_proto/inc/all_ops.h in the CANN software installation path.
    • For a custom operator, include the prototype definition header file xx.h in CANN software installation directory/opp/vendors/<vendor_name>/op_proto/inc.
  2. Create an operator instance.

    After the REG_OP macro is used to register the operator type, the operator type constructor explicit OpType(const char* name) is automatically generated, which is equivalent to defining a class op::xxx. Include the prototype header file and instantiate the class to build a graph. When building a graph, you can directly pass the operator name as the argument. For example:

    1
    		 
    auto softmax = op::SoftmaxV2("softmax")

    Note: The name of each operator in a graph must be unique.

  3. Set the operator inputs.

    The operator prototype defines the input names, input types, and data types supported by the operator. Operator inputs are classified into three types: optional inputs, required inputs, and dynamic inputs.

    Different APIs are used for setting different types of inputs.

    • For required inputs and optional inputs: Set by using set_input_inputName, as follows. 
      1
2
      		 
      auto softmax = op::SoftmaxV2("softmax")         // Create a SoftmaxV2 operator instance.
  .set_input_x(bias_add_3);                    // Set bias_add_3 as the input of SoftmaxV2 operator.
  4. Set the operator attributes.

    The operator prototype defines the operator attribute names and attribute types, as well as the supported data type, default value, and value range of each attribute. Based on the attribute types, operator attributes can be classified into required attributes (REQUIRED_ATTR) and optional attributes (ATTR). The value of a required attribute must be specified when the operator is defined. For an optional attribute, however, the default value will be used if the attribute is not specified.

    For attributes: Set by using the set_attr_attributeName API, as follows.

    1
2
3
4
5
    		 
    auto maxpool1 = op::MaxPool("MaxPool1")
.set_input_x(tanh1)
.set_attr_ksize({1, 1, 2, 1})                // Set the ksize attribute.
.set_attr_strides({1, 1, 2, 1})              // Set the strides attribute.
.set_attr_padding("SAME");                   // Set the padding attribute.

Connecting Operators with Edges

The edges connecting operators are classified into data edges and control edges. Data edges specify the operator input, while control edges specify the execution sequence of operators.

  1. Data edge expression

    If the upstream operator has only one output, pass the name of the upstream operator directly to the set_input_inputName call.

    1
2
3
4
5
6
    		 
    auto bias_add_3 = op::BiasAdd("bias_add_3")
  .set_input_x(matmul_2)
  .set_input_bias(bias_add_const_3)
  .set_attr_data_format("NCHW");
auto softmax = op::SoftmaxV2("Softmax")
  .set_input_x(bias_add_3);

    To distinguish between outputs in cases where the upstream operator has multiple outputs, you need to pass the upstream operator name as well as the specific output name or output index.

    The following provides an example of passing the operator name and output name of the upstream operator.

    1
2
3
4
    		 
    auto data = op::Data("data");
auto unique= op::Unique("unique").set_input_x(data);
auto softplus = op::Softplus("softplus").set_input_x(unique, "y");  // Create a Softplus operator and set the output y of the unique operator as Softplus' input.
auto sqrt = op::Sqrt("sqrt").set_input_x(unique, "idx");  // Create a Sqrt operator and set the output idx of the unique operator as Sqrt's input.

    The following provides an example of passing the operator name and output index of the upstream operator.

    1
2
3
4
    		 
    auto data = op::Data("data");
auto unique= op::Unique("unique").set_input_x(data);
auto softplus = op::Softplus("softplus").set_input_x(unique, 0);  // Create a Softplus operator and set the first output of the unique operator as Softplus' input.
auto sqrt = op::Sqrt("sqrt").set_input_x(unique, 1);  // Create a Sqrt operator and set the second output of the unique operator as Sqrt's input.
  2. Control edge expression

    Control edges can be added between operators to specify the execution sequence. As shown in the following figure, to execute Sqrt before the Softplus operator, a control edge needs to be added to the Softplus operator by using the AddControlInput API.

    A code sample is provided as follows.

    1
2
3
4
    		 
    auto data = op::Data("data");
auto unique= op::Unique("unique").set_input_x(data);
auto sqrt = op::Sqrt("sqrt").set_input_x(unique, "idx");  
auto softplus = op::Softplus("softplus").set_input_x(unique, "y").AddControlInput(sqrt);

Instantiating a Graph

After defining an operator, instantiate a graph by setting the input and output operators in a graph. The procedure is as follows:

  1. Include the header file. 
    1
    		 
    #include "graph.h"
  2. Create a graph object. 
    1
    		 
    Graph graph("IrGraph");

    For details about related APIs, see Graph Constructor and Destructor.

  3. Set the input and output operators in a graph by using the following API calls:

    In the following example, the Data operator and Softmax operator are set as the input operator and output operator of the graph, respectively.

    1
2
3
    		 
    std::vector<Operator> inputs{data}; // data indicates an operator object of the Data type.
std::vector<Operator> outputs{softmax}; // softmax indicates an operator object of the Softmax type.
graph.SetInputs(inputs).SetOutputs(outputs);

    To set more than one Data operator as the graph input, ensure that the inputs argument sequence is consistent with that specified by index of the Data operators. Otherwise, an error is reported during model generation. The following provides an example.

     1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
    		 
    // Prepare the first input.
auto shape_data0 = vector<int64_t>({1,17,2,2});
TensorDesc desc_data0(ge::Shape(shape_data0), FORMAT_ND, DT_FLOAT);
auto data0 = op::Data("data0").set_attr_index(0) ;   // Create operator data0 and set its index to 0.
data0.update_input_desc_x(desc_data0);         // Set the operator input description.
data0.update_output_desc_y(desc_data0);        // Set the operator output description.
// Prepare the second input.
auto shape_data1 = vector<int64_t>({1,5,2,2});
TensorDesc desc_data1(ge::Shape(shape_data1), FORMAT_ND, DT_FLOAT);
auto data1 = op::Data("data1").set_attr_index(1) ;   // Create operator data1 and set its index to 1.
data1.update_input_desc_x(desc_data1);         // Set the operator input description.
data1.update_output_desc_y(desc_data1);        // Set the operator output description.

// Set the input operators of the graph.
std::vector<Operator> inputs{data0, data1};
Parent topic: Preparations