Using C++ APIs to Construct a New Graph

The C++ graph development APIs of the GE supports two graph construction modes.

Using graph development APIs to construct a graph: You can add operators to a computational graph one at a time through graph development APIs (also called graph construction APIs) to construct a computational graph indicated by Ascend IR.
Using Parser APIs to parse the original model into a graph: Users can use Parser APIs to parse the framework model file (for example, in *.onnx or *.pb format) into the computational graph indicated by Ascend IR.

This section describes how to construct a graph according to the preceding two modes. After the graph is constructed, users can modify it as required and build it into an offline model adapted to Ascend AI Processor. The process is as follows.

Process description:

Before graph construction, set up the environment, install the CANN package, and analyze the network structure. If there are unsupported operators, develop custom operators and deploy them in the hardware environment.
1. If you use graph development APIs to construct a graph, determine the following information based on the source network:
  1. Check out the operators on the network and the inputs, outputs, and attributes of each operator.
  2. Check out the connections between operators on the network.
  3. Check whether the operators on the source network are supported by Ascend AI Processor by referring to "Ascend IR Operator Specifications". If the operators are not supported or the actual requirements are not met, you can use any of the following methods to customize operators:
    Customize an Ascend C operator by referring to Ascend C Operator Development Guide and deploy the operator in the hardware environment.
2. If Parser APIs are used to parse the original model into a graph, check whether the operators on the source network are supported by Ascend AI Processor by referring to "Ascend IR Operator Specifications". If the operators are not supported or the actual requirements are not met, you can use any of the following methods to customize operators:
  Customize an Ascend C operator by referring to Ascend C Operator Development Guide and deploy the operator in the hardware environment.
Construct a graph. You can use a graph development API to construct a new graph or use a Parser API to parse the original model into a graph.
Modify the graph. If you want to optimize the graph structure, you can modify the constructed graph to the expected structure.
Build and run the graph. You can build the modified graph into an offline model that adapts to the Ascend AI Processor, and load the model through acl APIs for inference. Alternatively, you can also directly run the graph to obtain the graph execution result.

Sample Obtaining

Click the link to obtain the sample code. The directory structure is as follows:

├── src 
│   ├──main.cpp                //Implementation file
├── Makefile                    //Building scripts
├── CMakeLists.txt              ///Building scripts
├── data          
│   ├──data_generate.py        //Data, for example, weight and offset data, required for new graph construction using graph development APIs
│   ├──tensorflow_generate.py  //File used to generate a TensorFlow model (.pb) for parsing a TensorFlow original model into a graph using a Parser API
│   ├──caffe_generate.py       //File used to generate a Caffe model (.pbtxt) and the weight file (.caffemodel) for parsing a Caffe original model into a graph using a Parser API
├── scripts 
│   ├──host_version.conf       //Version configuration
│   ├──testcase_300.sh         //Test scripts

Main functions of the sample:

Graph construction
- Using graph development APIs to construct a new graph
  The structure of the graph constructed using the sample is as follows.
- Using Parser APIs to parse the original model into a graph. The following uses TensorFlow as an example.
  In this sample, the structure of the source TensorFlow model is as follows.
  
  The parsed graph structure is as follows.
Graph modification. The following uses the graph obtained by parsing the TensorFlow model using Parser APIs as an example.
The modified graph structure is as follows. The following example adds operator Abs between operators Add and Const.
Graph building. The previously obtained graph is built into an offline model.

Building and Running

Set up the environment.
- Set up the environment as instructed in Environment Setup and configure environment variables.
- To parse the TensorFlow original model into a graph, set up the TensorFlow 1.15.0 environment. The following is an example of installing TensorFlow 1.15.0 on the x86 architecture as user root:
```
pip3 install tensorflow-cpu==1.15
```
  If the installation is performed by a non-root user, add the --user parameter to the end of the preceding command. If the aarch64 architecture is used, see TensorFlow 1.15 Model Porting Guide > "Environment Setup" > "Installing TensorFlow 1.15".
Prepare graph construction data.
- Using graph development APIs to construct a new graph
  Run the data generation scripts in the data directory.
```
python3 data_generate.py
```
  After the command execution is complete, data in .bin format is generated in the data directory. Data such as weights and offsets will be read from this file during model building.
- Using Parser APIs to parse the original model into a graph
  Run the original model generation scripts in the data directory.
```
python3 tensorflow_generate.py
```
  After the command execution is complete, a model file in .pb format, named tf_test.pb, is generated in the data directory. This model will be parsed as the original model later.
Build the code.
1. (Optional) Open the Makefile file and change the value of ASCEND_PATH to the CANN software installation directory /cann. For example, change the value to /usr/local/Ascend/cann for user root.
```
ifeq (${ASCEND_INSTALL_PATH},)    
     ASCEND_PATH := /usr/local/Ascend/cann
```
2. Start building.
  Start building in the directory of the Makefile file.
```
make clean && make ir_build
```
  After the building is complete, the executable file ir_build is generated in the out folder of the current path.

Run the code.

Using graph development APIs to construct a new graph

Run the following command in the out folder:

./ir_build <soc_version> gen

If the following information is displayed, the code running is successful and an offline model file *.om is generated in the same directory.

          
               ========== Generate Graph1 Success!==========  
Build Model1 SUCCESS! 
Save Offline Model1 SUCCESS!

Using Parser APIs to parse the original model into a graph

Run the following command in the out folder:

./ir_build <soc_version> tf

If the following information is displayed, the code running is successful and an offline model file *.om is generated in the same directory.

          
               ========== Generate graph from tensorflow origin model success.========== 
Modify Graph Start. 
Find src node: const. 
Find dst node: add. 
Modify Graph Success. 
========== Modify tensorflow origin graph success.========== 
Build Model1 SUCCESS! 
Save Offline Model1 SUCCESS!

To query the value of <soc_version>, perform the following:

For the following products: Run the npu-smi info command on the server where Ascend AI Processor is installed to obtain the Name information. The actual value is AscendName. For example, if Name is xxxyy, the actual value is Ascendxxxyy.
Atlas A2 training products / Atlas A2 inference products

Atlas 200I/500 A2 inference products

Atlas inference products

Atlas training products
For the following products: Run the npu-smi info -t board -i id -c chip_id command on the server where Ascend AI Processor is installed to obtain the Chip Name and NPU Name information. The actual value is Chip Name_NPU Name. For example, if the value of Chip Name is Ascendxxx and the value of NPU Name is 1234, the actual value is Ascendxxx_1234. Note that:
- id: device ID, which is the NPU ID obtained by running the npu-smi info -l command.
- chip_id: chip ID, which is obtained by running the npu-smi info -m command.
Atlas A3 training products / Atlas A3 inference products

Perform subsequent operations.
Call acl APIs to load the obtained *.om offline model for model inference. For details about how to load a model and perform inference, see "Inference App Development Workflow".

Code Parsing

If you want to learn about the code implementation logic after building and running the quick start sample, see the following code analysis. This section describes the code logic based on what each function does. To facilitate code reading and understanding, the functions and related variables are described together.

Open the src/main.cpp file, start from the main function to understand the code concatenation logic of the entire sample, and then explore the implementation of the custom functions.

Concatenate the code logic of the entire graph construction using the main function.

        
         
           
           
             int main(int argc, char* argv[])
{
    // 1. Construct a graph.
    Graph graph1("IrGraph1");
       // 1.1 Use graph development APIs to construct a new graph.
        ret = GenGraph(graph1);

       // 1.2 Use Parser APIs to parse the original model into a graph.
        std::string tfPath = "../data/tf_test.pb";
        std::map<AscendString, AscendString> parser_options;
        tfStatus = ge::aclgrphParseTensorFlow(tfPath.c_str(), parser_options, graph1);

       // 2. Modify the graph. The following uses the graph obtained by parsing the TensorFlow model as an example.
        ModifyGraph(graph1) 

    // 3. Initialize the system and allocate resources.
    std::map<AscendString, AscendString> global_options = {
        {AscendString(ge::ir_option::SOC_VERSION), AscendString(argv[kSocVersion])}  ,
    };
    auto status = aclgrphBuildInitialize(global_options);

    // 4. Build the graph.
    ModelBufferData model1;
    std::map<AscendString, AscendString> options;
    PrepareOptions(options);
    status = aclgrphBuildModel(graph1, options, model1);

    // 5. Save the model in the memory buffer as an offline model file.
    status = aclgrphSaveModel("ir_build_sample1", model1);
 
    // 6. Destroy resources.
    aclgrphBuildFinalize();

}

            

          

        
       

Include the depended header files.

        
             #include <string.h>
#include "tensorflow_parser.h"
#include "graph.h"
#include "types.h"
#include "tensor.h"
#include "attr_value.h"
#include "ge_error_codes.h"
#include "ge_api_types.h"
#include "ge_ir_build.h"
#include "all_ops.h"
... ...
using namespace std;
using namespace ge;
using ge::Operator;

Construct a graph.

Define operators using APIs derived from operator prototypes and connect the operators. The sample code is as follows:

         
          
            
            
              bool GenGraph(Graph& graph)
{
    auto shape_data = vector<int64_t>({ 1,1,28,28 });
    TensorDesc desc_data(ge::Shape(shape_data), FORMAT_ND, DT_FLOAT16);
    // Create a Data operator instance.
    auto data = op::Data("data");
    data.update_input_desc_x(desc_data);
    data.update_output_desc_y(desc_data);

    // Create an Add operator instance and connect it to the Data operator.
    auto add = op::Add("add")
        .set_input_x1(data)
        .set_input_x2(data);

    // Create an AscendQuant operator instance and connect it to the Data operator.
    auto quant = op::AscendQuant("quant")
        .set_input_x(data)
        .set_attr_scale(1.0)
        .set_attr_offset(0.0);
 
    auto conv_weight = op::Const("Conv2D/weight")
        .set_attr_value(weight_tensor);
    // Create an Conv2D operator instance and connect it to the AscendQuant operator.
    auto conv2d = op::Conv2D("Conv2d1")
        .set_input_x(quant)
        .set_input_filter(conv_weight)
        .set_attr_strides({ 1, 1, 1, 1 })
        .set_attr_pads({ 0, 1, 0, 1 })
        .set_attr_dilations({ 1, 1, 1, 1 });
        ... ...

    auto dequant_scale = op::Const("dequant_scale")
        .set_attr_value(dequant_tensor);
    // Create an AscendDequant operator instance and connect it to the Conv2D operator.
    auto dequant = op::AscendDequant("dequant")
        .set_input_x(conv2d)
        .set_input_deq_scale(dequant_scale);
        ... ...

    auto bias_weight_1 = op::Const("Bias/weight_1")
        .set_attr_value(weight_bias_add_tensor_1);
    // Create a BiasAdd operator instance and connect it to the AscendDequant operator.
    auto bias_add_1 = op::BiasAdd("bias_add_1")
        .set_input_x(dequant)
        .set_input_bias(bias_weight_1)
        .set_attr_data_format("NCHW");
        ... ...

    auto dynamic_const = op::Const("dynamic_const").set_attr_value(dynamic_const_tensor);
    // Create a Reshape operator instance and connect it to the BiasAdd operator.
    auto reshape = op::Reshape("Reshape")
        .set_input_x(bias_add_1)
        .set_input_shape(dynamic_const);
        ... ...

    auto matmul_weight_1 = op::Const("dense/kernel")
        .set_attr_value(matmul_weight_tensor_1);
    // Create a MatMul operator instance and connect it to the Reshape operator.
    auto matmul_1 = op::MatMul("MatMul_1")
        .set_input_x1(reshape)
        .set_input_x2(matmul_weight_1);
        ... ...

    auto bias_add_const_1 = op::Const("dense/bias")
        .set_attr_value(bias_add_const_tensor_1);
    // Create a BiasAdd operator instance and connect it to the MatMul operator.
    auto bias_add_2 = op::BiasAdd("bias_add_2")
        .set_input_x(matmul_1)
        .set_input_bias(bias_add_const_1)
        .set_attr_data_format("NCHW");
    // Create a Relu6 operator instance and connect it to the BiasAdd operator.
    auto relu6 = op::Relu6("relu6")
        .set_input_x(bias_add_2);
        ... ...

    auto matmul_weight_2 = op::Const("OutputLayer/kernel")
        .set_attr_value(matmul_weight_tensor_2);
    // Create a MatMul operator instance and connect it to the Relu6 operator.
    auto matmul_2 = op::MatMul("MatMul_2")
        .set_input_x1(relu6)
        .set_input_x2(matmul_weight_2);
        ... ...

    auto bias_add_const_3 = op::Const("OutputLayer/bias")
        .set_attr_value(bias_add_const_tensor_3);
    // Create a BiasAdd operator instance and connect it to the MatMul operator.
    auto bias_add_3 = op::BiasAdd("bias_add_3")
        .set_input_x_by_name(matmul_2, "y")
        .set_input_bias_by_name(bias_add_const_3, "y")
        .set_attr_data_format("NCHW");
    // Create a SoftmaxV2 operator instance and connect it to the BiasAdd operator.
    auto softmax = op::SoftmaxV2("Softmax")
        .set_input_x_by_name(bias_add_3, "y");
    std::vector<Operator> inputs{ data };

    std::vector<Operator> outputs{ softmax, add };
    std::vector<std::pair<ge::Operator, std::string>> outputs_with_name = {{softmax, "y"}};
    // Call APIs to set the inputs and outputs of the graph.
    graph.SetInputs(inputs).SetOutputs(outputs);
    return true;
}

             

           

         
        

Modify the graph. The following uses the graph obtained by parsing the TensorFlow model as an example.

Insert the Abs operator between the Add operator and the Const operator.

         
              bool ModifyGraph(Graph &graph) {

    const std::string CONST = "const1";
    const std::string ADD = "input3_add";
    GNode src_node;
    GNode dst_node;
    // Obtain all nodes in the graph.
    std::vector<GNode> nodes = graph.GetAllNodes();
    graphStatus ret = GRAPH_FAILED;
    for (auto &node : nodes) {
        ge::AscendString name;
        // Obtain the operator name in the graph.
        ret = node.GetName(name);

        std::string node_name(name.GetString());
    }
    // Removes a specified edge from the graph.
    ret = graph.RemoveEdge(src_node, 0, dst_node, 1);

    auto abs = op::Abs("input3_abs");
    
// Add the abs node to the graph.
    GNode node_abs = graph.AddNodeByOp(abs);
   
   // Add a data edge to connect the added node.
    ret = graph.AddDataEdge(src_node, 0, node_abs, 0);

    ret = graph.AddDataEdge(node_abs, 0, dst_node, 1);
    return true;
}

Build the graph.
After constructing and modifying the graph, call APIs to build the graph. The following figure shows the API call process. For details, see the main() function.

Parent topic: Quick Start