Using Parser APIs to Parse the Original Model into a Graph

Overview

Currently, open-source deep learning frameworks (such as TensorFlow, PyTorch, and Caffe) define models in different formats. For example, TensorFlow uses user-defined .pb to describe static shape graphs and models while PyTorch uses ONNX for the same purpose. Therefore, a unified framework-based Parser layer is introduced into AI processor, which shields the differences at the framework level by converting open-source models into representations acceptable by CANN.

The following APIs are involved:

• aclgrphParseTensorFlow: parses a TensorFlow model.
• aclgrphParseCaffe: parses a Caffe model.

  Atlas A2 training products / Atlas A2 inference products : The Caffe framework is not supported. Development for Caffe on this product form has ceased, and therefore its functionality is not guaranteed.

  Atlas A3 training products / Atlas A3 inference products : The Caffe framework is not supported. Development for Caffe on this product form has ceased, and therefore its functionality is not guaranteed.

  Ascend 950PR / Ascend 950DT : The Caffe framework is not supported. Development for Caffe on this product form has ceased, and therefore its functionality is not guaranteed.

• aclgrphParseONNX: parses an ONNX model.
• aclgrphParseONNXFromMem: parses an ONNX model that is loaded to memory.

Currently, the Parser layer enables custom OpParser and custom TensorFlow scope fusion patterns, which allows flexible adaptation to different frameworks.

• Custom OpParser:

  Allows you to directly map the original open-source operators to Ascend C or TBE operators adapted to CANN. For details, see Programming Guide > Appendixes > AI Framework Operator Adaptation in Ascend C Operator Development or Operator Adaptation in TBE & AI CPU Operator Development. Select one type of adapted operators.

• Custom TensorFlow scope fusion patterns: fuse operators in the TensorFlow computational graph, when possible, to facilitate high-performance computing and utilize the hardware acceleration capability. For details, see TensorFlow Parser Scope Fusion Pattern Development.

This note does not apply to Ascend 950PR / Ascend 950DT .

If the number of dimensions in the shape of a tensor is different from that in the format when the original model is converted to a graph, the current dimensions are interpreted according to the rules in the following table.

For example, if the shape is one-dimensional ([16]) and the format is 4D (for example, NHWC), the first dimension of the shape can be considered as the C axis, and the dimensions of other axes need to be expanded. After expansion, the format is [1,1,1,16].

If the shape is 2D ([16,16]) and the format is 4D (for example, NHWC), the two dimensions of the shape can be considered as the HW axis, and the dimensions of other axes need to be expanded. After expansion, the format is [1,16,16,1].

TensorFlow-based Model Parsing

Include the header files.

1

#include "tensorflow_parser.h"

1
2
3

std::string tfPath = "../data/tf_test.pb";
ge::Graph graph1;
auto tfStatus = ge::aclgrphParseTensorFlow(tfPath.c_str(),graph1);

Set parser_params as required.

1
2
3
4
5
6

std::string tfPath = "../data/tf_test.pb";
std::map<ge::AscendString, ge::AscendString> parser_params = {
            {ge::AscendString(ge::ir_option::INPUT_FP16_NODES), ge::AscendString("input1;input2")},
            {ge::AscendString(ge::ir_option::OUTPUT), ge::AscendString("newIssue")}};
ge::Graph graph1;
auto tfStatus = ge::aclgrphParseTensorFlow(tfPath.c_str(), parser_params, graph1);

Caffe-based Model Parsing

Include the header files.

1

#include "caffe_parser.h"

1
2
3
4

std::string caffePath = "../data/caffe_test.prototxt";
std::string weight = "../data/caffe_test.caffemodel";
ge::Graph graph1;
auto caffeStatus = ge::aclgrphParseCaffe(caffePath.c_str(), weight.c_str(), graph1);

Set parser_params as required.

1
2
3
4
5
6
7

std::string caffePath = "../data/caffe_test.prototxt";
std::string weight = "../data/caffe_test.caffemodel";
std::map<ge::AscendString, ge::AscendString> parser_params = {
            {ge::AscendString(ge::ir_option::INPUT_FP16_NODES), ge::AscendString("input1;input2")},
            {ge::AscendString(ge::ir_option::OUTPUT), ge::AscendString("newIssue")}};
ge::Graph graph1;
auto caffeStatus = ge::aclgrphParseCaffe(caffePath.c_str(), weight.c_str(), parser_params, graph1);

ONNX-based Model Parsing

Include the header files.

1

#include "onnx_parser.h"

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34

// Open an ONNX model file in binary mode.
FILE *pFile = fopen("./onnx/resnet101.onnx", "rb" );
if(pFile==NULL)
{
    fputs("File error",stderr);
    exit(1);
}

// Obtain the file size.
fseek(pFile, 0, SEEK_END);
long lSize = ftell(pFile);
rewind(pFile);

// Allocate a sufficient memory buffer.
char *buffer =(char*) malloc(sizeof(char)*lSize);
if(buffer == NULL)
{
    fputs("Memory error", stderr); 
    exit(2);
}
// Read the file content to the buffer.
size_t result = fread(buffer, 1, lSize, pFile);
if(result != lSize)
{
    fputs("Reading error", stderr);
    exit(3);
}
// Prepare the parameters required for the ONNX model parsing.
std::map<ge::AscendString, ge::AscendString> parser_params= {
            {ge::AscendString(ge::ir_option::INPUT_FP16_NODES), ge::AscendString("input1;input2")},
            {ge::AscendString(ge::ir_option::OUTPUT), ge::AscendString("newIssue")}};
// Parse the ONNX model.
ge::Graph graph1;
auto onnxStatus = ge::aclgrphParseONNXFromMem(buffer, result, parser_params, graph1);

Call aclgrphParseONNX to parse an ONNX model into a graph. The result graph is saved in the memory buffer. Set parser_params as required.

1
2
3
4
5
6

std::string onnxPath = "../data/onnx_test.onnx";
std::map<ge::AscendString, ge::AscendString> parser_params= {
            {ge::AscendString(ge::ir_option::INPUT_FP16_NODES), ge::AscendString("input1;input2")},
            {ge::AscendString(ge::ir_option::OUTPUT), ge::AscendString("newIssue")}};
ge::Graph graph1;
auto onnxStatus = ge::aclgrphParseONNX(onnxPath.c_str(), parser_params, graph1);