Graph Parsing from Original Model

Overview

Currently, open-source deep learning frameworks (such as TensorFlow, PyTorch, and Caffe) define models in different formats. For example, TensorFlow uses user-defined .pb to describe static shape graphs and models while PyTorch uses ONNX for the same purpose. Therefore, a unified framework-based Parser layer is introduced into Ascend AI Processor, which shields the differences at the framework level by converting open-source models into representations acceptable by CANN.

The following APIs are involved:

• aclgrphParseTensorFlow: parses a TensorFlow model.
• aclgrphParseCaffe: parses a Caffe model.
• aclgrphParseONNX: parses an ONNX model.
• aclgrphParseONNXFromMem: parses an ONNX model that is loaded to memory.

Currently, the Parser layer enables custom OpParser and custom TensorFlow scope fusion patterns, which allows flexible adaptation to different frameworks.

• Custom OpParser:
  • Allows you to directly map the original open-source operators to Ascend C operators adapted to CANN. For details, see AI Framework Operator Adaptation.
  • Allows you to directly map the original open-source operators to TBE operators adapted to CANN. For details, see Operator Adaptation.
• Custom TensorFlow scope fusion patterns: fuse operators in the TensorFlow computational graph, when possible, to facilitate high-performance computing and utilize the hardware acceleration capability. For details, see TensorFlow Parser Scope Fusion Pattern Developer Guide.

TensorFlow-Based Model Parsing

Include the header files.

1

#include "tensorflow_parser.h"

1
2
3

std::string tfPath = "../data/tf_test.pb";
ge::Graph graph1;
auto tfStatus = ge::aclgrphParseTensorFlow(tfPath.c_str(),graph1);

Set parser_params as required.

1
2
3
4
5
6

std::string tfPath = "../data/tf_test.pb";
std::map<ge::AscendString, ge::AscendString> parser_params= {
            {ge::AscendString(ge::ir_option::INPUT_FP16_NODES), ge::AscendString("input1;input2")},
            {ge::AscendString(ge::ir_option::OUTPUT), ge::AscendString("newIssue")}};
ge::Graph graph1;
auto tfStatus = ge::aclgrphParseTensorFlow(tfPath.c_str(), parser_params, graph1);

Caffe-Based Model Parsing

Include the header files.

1

#include "caffe_parser.h"

1
2
3
4

std::string caffePath = "../data/caffe_test.prototxt";
std::string weight = "../data/caffe_test.caffemodel";
ge::Graph graph1;
auto caffeStatus = ge::aclgrphParseCaffe(caffePath.c_str(), weight.c_str(), graph1);

Set parser_params as required.

1
2
3
4
5
6
7

std::string caffePath = "../data/caffe_test.prototxt";
std::string weight = "../data/caffe_test.caffemodel";
std::map<ge::AscendString, ge::AscendString> parser_params= {
            {ge::AscendString(ge::ir_option::INPUT_FP16_NODES), ge::AscendString("input1;input2")},
            {ge::AscendString(ge::ir_option::OUTPUT), ge::AscendString("newIssue")}};
ge::Graph graph1;
auto caffeStatus = ge::aclgrphParseCaffe(caffePath.c_str(), weight.c_str(), parser_params, graph1);

ONNX-Based Model Parsing

Include the header files.

1

#include "onnx_parser.h"

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32

/* read file in binary format */
FILE *pFile = fopen("./onnx/resnet101.onnx", "rb" );
if(pFile==NULL)
{
    fputs("File error",stderr);
    exit(1);
}

/* get the size of the file */
fseek(pFile, 0, SEEK_END);
long lSize = ftell(pFile);
rewind(pFile);

/* assign memory buffer for the file*/
char *buffer =(char*) malloc(sizeof(char)*lSize);
if(buffer == NULL)
{
    fputs("Memory error", stderr); 
    exit(2);
}
/* copy the file to buffer */
size_t result = fread(buffer, 1, lSize, pFile);
if(result != lSize)
{
    fputs("Reading error", stderr);
    exit(3);
}
std::map<ge::AscendString, ge::AscendString> parser_params= {
            {ge::AscendString(ge::ir_option::INPUT_FP16_NODES), ge::AscendString("input1;input2")},
            {ge::AscendString(ge::ir_option::OUTPUT), ge::AscendString("newIssue")}};
ge::Graph graph1;
auto onnxStatus = ge::aclgrphParseONNXFromMem(buffer, result, parser_params, graph1);

Call aclgrphParseONNX to parse an ONNX model into a graph. The result graph is saved in the memory buffer. Set parser_params as required.

1
2
3
4
5
6

std::string onnxPath = "../data/onnx_test.onnx";
std::map<ge::AscendString, ge::AscendString> parser_params= {
            {ge::AscendString(ge::ir_option::INPUT_FP16_NODES), ge::AscendString("input1;input2")},
            {ge::AscendString(ge::ir_option::OUTPUT), ge::AscendString("newIssue")}};
ge::Graph graph1;
auto onnxStatus = ge::aclgrphParseONNX(onnxPath.c_str(), parser_params, graph1);