Executing an ONNX or PB model in graph mode

GE provides the function of parsing models exported from the front-end framework. If you have obtained models (*.onnx or *.pb) exported from the PyTorch or TensorFlow framework, you can use the ATC command line tool provided by GE or the parser API of the C++ language, the two types of models are converted into an offline model that adapts to Ascend AI Processor and executed based on the graph mode.

The .air model exported from the MindSpore framework can also be converted into an offline model adapted to Ascend AI Processor by using the ATC command line tool, and then loaded and executed by using the AscendCL API.

The following describes how to execute a model exported from the PyTorch or TensorFlow framework in graph mode.

Parsing and Executing a Model by Using the ATC Command Line Tool

Use the ATC command line tool to convert the .pb model exported from the TensorFlow or .onnx model exported from the PyTorch framework into an .om offline model that adapts to Ascend AI Processor.
The following is an example of using the ATC tool to convert the .onnx model:
```
atc --model=$HOME/module/resnet50*.onnx --framework=5 --output=$HOME/module/out/onnx_resnet50 --soc_version=<soc_version>
```
The following is an example of using the ATC tool to convert a .pb model:
```
atc --model=$HOME/module/resnet50_tensorflow*.pb --framework=3 --output=$HOME/module/out/tf_resnet50 --soc_version=<soc_version>
```
The key parameters are described as follows. For details about the parameters supported by ATC, see ATC Instructions.
- --model: path (including the file name) of the original network model.
- --framework: framework type of the original network model. 3: TensorFlow; 5: ONNX.
- --output: path (including the file name) for storing the generated offline model after conversion, for example, $HOME/module/out/tf_resnet50, where $HOME/module/out/ is the path for storing the generated offline model, and tf_resnet50.om is the name of the generated offline model.
- --soc_version: Ascend AI Processor version.
Develop applications based on the AscendCL APIs, load the generated offline model after conversion, and perform model inference.
For details about how to load and infer a model, see Inference App Development Workflow .

Parsing and Executing a Model by Using the Parser API of the C++ Language

For different framework models, GE provides parsing APIs to parse and convert the models into those adapted to Ascend. The following shows the key parsing APIs and calling examples of different framework models.

aclgrphParseONNX: parses the ONNX model.

         
              #include "onnx_parser.h"
std::string onnxPath = "../data/onnx_test.onnx";
std::map<ge::AscendString, ge::AscendString> parser_params= {
            {ge::AscendString(ge::ir_option::INPUT_FP16_NODES), ge::AscendString("input1;input2")},
            {ge::AscendString(ge::ir_option::OUTPUT), ge::AscendString("newIssue")}};
ge::Graph graph1;
auto onnxStatus = ge::aclgrphParseONNX(onnxPath.c_str(), parser_params, graph1);

aclgrphParseONNXFromMem: parses an ONNX model that is loaded to memory.

         
          
            
            
              #include "onnx_parser.h"

FILE *pFile = fopen("./onnx/resnet101.onnx", "rb" );
if(pFile==NULL)
{
    fputs("File error",stderr);
    exit(1);
}

/* get the size of the file */
fseek(pFile, 0, SEEK_END);
long lSize = ftell(pFile);
rewind(pFile);

/* assign memory buffer for the file*/
char *buffer =(char*) malloc(sizeof(char)*lSize);
if(buffer == NULL)
{
    fputs("Memory error", stderr); 
    exit(2);
}
/* copy the file to buffer */
size_t result = fread(buffer, 1, lSize, pFile);
if(result != lSize)
{
    fputs("Reading error", stderr);
    exit(3);
}
std::map<ge::AscendString, ge::AscendString> parser_params= {
            {ge::AscendString(ge::ir_option::INPUT_FP16_NODES), ge::AscendString("input1;input2")},
            {ge::AscendString(ge::ir_option::OUTPUT), ge::AscendString("newIssue")}};
ge::Graph graph1;
auto onnxStatus = ge::aclgrphParseONNXFromMem(buffer, result, parser_params, graph1);

             

           

         
        

aclgrphParseTensorFlow: parses a TensorFlow model.

A code example is shown below:

          
               #include "tensorflow_parser.h"
std::string tfPath = "../data/tf_test.pb";
ge::Graph graph1;
auto tfStatus = ge::aclgrphParseTensorFlow(tfPath.c_str(),graph1);

When calling the parsing API, you can also use the parser_params parameter to configure extended parameters. For details, see Graph Parsing from Original Model .

After the original framework model is parsed, the GE graph (graph1 object in the preceding code example) is obtained. In this case, the graph is stored in the memory buffer. You can directly compile and run the graph. For details about the graph compilation and running process, see the next section.