Using Parser APIs to Parse the Original Model into a Graph

Overview

Currently, open-source deep learning frameworks (such as TensorFlow, PyTorch, and Caffe) define models in different formats. For example, TensorFlow uses user-defined .pb to describe static shape graphs and models while PyTorch uses ONNX for the same purpose. Therefore, a unified framework-based Parser layer is introduced into Ascend AI Processor, which shields the differences at the framework level by converting open-source models into representations acceptable by CANN.

The main APIs are as follows.

• aclgrphParseTensorFlow: parses a TensorFlow model.
• aclgrphParseCaffe: parses a Caffe model.

  The Atlas A2 training products / Atlas A2 inference products do not support the Caffe framework.

  The Atlas A3 training products / Atlas A3 inference products do not support the Caffe framework.

• aclgrphParseONNX: parses an ONNX model.
• aclgrphParseONNXFromMem: parses an ONNX model that is loaded to memory.

Currently, the Parser layer enables custom OpParser and custom TensorFlow scope fusion patterns, which allows flexible adaptation to different frameworks.

• Custom OpParser:

  Allows you to directly map the original open-source operators to Ascend C operators adapted to CANN. For details, see AI Framework Operator Adaptation.

• Custom TensorFlow scope fusion patterns: fuse operators in the TensorFlow computational graph, when possible, to facilitate high-performance computing and utilize the hardware acceleration capability. For details, see TensorFlow Parser Scope Fusion Pattern Developer Guide.

If the number of dimensions in the shape of a tensor is different from that in the format when the original model is converted to a graph, the current dimensions are interpreted according to the rules in following table.

For example, if the shape is one-dimensional ([16]) and the format is 4D (for example, NHWC), the first dimension of the shape can be considered as the C axis, and the dimensions of other axes need to be expanded. After expansion, the format is [1,1,1,16].

If the shape is 2D ([16,16]) and the format is 4D (for example, NHWC), the two dimensions of the shape can be considered as the HW axis, and the dimensions of other axes need to be expanded. After expansion, the format is [1,16,16,1].

TensorFlow-based Model Parsing

Include the header file.

1

#include "tensorflow_parser.h"

1
2
3

std::string tfPath = "../data/tf_test.pb";
ge::Graph graph1;
auto tfStatus = ge::aclgrphParseTensorFlow(tfPath.c_str(),graph1);

Set parser_params as required.

1
2
3
4
5
6

std::string tfPath = "../data/tf_test.pb";
std::map<ge::AscendString, ge::AscendString> parser_params = {
            {ge::AscendString(ge::ir_option::INPUT_FP16_NODES), ge::AscendString("input1;input2")},
            {ge::AscendString(ge::ir_option::OUTPUT), ge::AscendString("newIssue")}};
ge::Graph graph1;
auto tfStatus = ge::aclgrphParseTensorFlow(tfPath.c_str(), parser_params, graph1);

Caffe-based Model Parsing

Include the header file.

1

#include "caffe_parser.h"

1
2
3
4

std::string caffePath = "../data/caffe_test.prototxt";
std::string weight = "../data/caffe_test.caffemodel";
ge::Graph graph1;
auto caffeStatus = ge::aclgrphParseCaffe(caffePath.c_str(), weight.c_str(), graph1);

Set parser_params as required.

1
2
3
4
5
6
7

std::string caffePath = "../data/caffe_test.prototxt";
std::string weight = "../data/caffe_test.caffemodel";
std::map<ge::AscendString, ge::AscendString> parser_params = {
            {ge::AscendString(ge::ir_option::INPUT_FP16_NODES), ge::AscendString("input1;input2")},
            {ge::AscendString(ge::ir_option::OUTPUT), ge::AscendString("newIssue")}};
ge::Graph graph1;
auto caffeStatus = ge::aclgrphParseCaffe(caffePath.c_str(), weight.c_str(), parser_params, graph1);

ONNX-based Model Parsing

Include the header file.

1

#include "onnx_parser.h"

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34

// Open an ONNX model file in binary mode.
FILE *pFile = fopen("./onnx/resnet101.onnx", "rb" );
if(pFile==NULL)
{
    fputs("File error",stderr);
    exit(1);
}

// Obtain the file size.
fseek(pFile, 0, SEEK_END);
long lSize = ftell(pFile);
rewind(pFile);

// Allocate a sufficient memory buffer.
char *buffer =(char*) malloc(sizeof(char)*lSize);
if(buffer == NULL)
{
    fputs("Memory error", stderr); 
    exit(2);
}
// Read the file content to the buffer.
size_t result = fread(buffer, 1, lSize, pFile);
if(result != lSize)
{
    fputs("Reading error", stderr);
    exit(3);
}
// Prepare the parameters required for the ONNX model parsing.
std::map<ge::AscendString, ge::AscendString> parser_params= {
            {ge::AscendString(ge::ir_option::INPUT_FP16_NODES), ge::AscendString("input1;input2")},
            {ge::AscendString(ge::ir_option::OUTPUT), ge::AscendString("newIssue")}};
// Parse the ONNX model.
ge::Graph graph1;
auto onnxStatus = ge::aclgrphParseONNXFromMem(buffer, result, parser_params, graph1);

Call aclgrphParseONNX to parse an ONNX original model into a graph. The result graph is saved in the memory buffer. Set parser_params as required.

1
2
3
4
5
6

std::string onnxPath = "../data/onnx_test.onnx";
std::map<ge::AscendString, ge::AscendString> parser_params= {
            {ge::AscendString(ge::ir_option::INPUT_FP16_NODES), ge::AscendString("input1;input2")},
            {ge::AscendString(ge::ir_option::OUTPUT), ge::AscendString("newIssue")}};
ge::Graph graph1;
auto onnxStatus = ge::aclgrphParseONNX(onnxPath.c_str(), parser_params, graph1);