Compiling and Executing FlowGraph

Description

This section describes how to compile and execute the constructed FlowGraph to generate a result. The main APIs are as follows.

1. GEInitialize: initializes the system and allocates system resources. (This API can be called before graph construction.)
2. Session constructor: creates a Session object and allocates Session resources.
3. FlowGraph: adds a defined graph to the Session object.
4. RunGraph, or FeedDataFlowGraph (Feeding All Inputs) and FetchDataFlowGraph (Obtaining All Output Data)/FetchDataFlowGraph (Fetching Output Data by Index): executes a graph.
5. GEFinalize: releases system resources.

For details about GEInitialize, Session constructor, AddGraph, RunGraph, and GEFinalize, see Graph Mode Development Guide.

When the DataFlow development framework is used, the NN model is executed in saturation mode. The computing accuracy in this mode may be inaccurate. In addition, this mode is used only for compatibility with earlier versions and will not evolve in the future.

Development Example

1. Include the header file.

   1	#include "ge_api.h"

2. Allocate system resources.

   After a graph is defined, call GEInitialize to initialize the system (or call it before defining a graph) and allocate system resources. A code example is as follows:

   1 2 3 4 5 6

   std::map<AscendString, AscendString>config = {{"ge.exec.deviceId", "0"}, {"ge.exec.logicalDeviceClusterDeployMode", "SINGLE"}, {"ge.exec.logicalDeviceId", "[0:0]"}, {"ge.graphRunMode", "1"}, {"ge.exec.precision_mode", "allow_fp32_to_fp16"}}; Status ret = ge::GEInitialize(config);

   Set the GE initialization configuration by using config. The parameters in the preceding configuration (ge.exec.deviceId, ge.graphRunMode, and ge.exec.precision_mode) are for specifying the device where a GE instance runs, for specifying the graph execution mode (set to 0 for online inference and 1 for training), and for specifying the operator precision mode, respectively. For details about the options, see ""Options"".

   

   Some UDFs do not support load sharing. Therefore, {"ge.exec.logicalDeviceClusterDeployMode", "SINGLE"} and {"ge.exec.logicalDeviceId", "[0:0]"} need to be added during GE initialization. The value of logicalDeviceId can be [0:0] or [0:1]. You can refer to Specifying the Deployment Location of the DataFlow Node to implement multi-instance deployment if required. logicalDeviceId is described as follows:

   Device on which the model is deployed when logicalDeviceClusterDeployMode is set to SINGLE.

   Format: [node_id:device_id]

   • node_id: logical ID of the Ascend AI Processor, starting from 0, indicating the sequence number of the device in the resource configuration file.
   • device_id: physical ID of the Ascend AI Processor.
3. Add a graph object and execute the graph.
   To execute a graph, create a Session object, call the AddGraph API to add the graph, and then call the RunGraph API to execute the graph. A code example is as follows:

   1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26

   std::map <AscendString, AscendString> options; ge::Session *session = new Session(options); if(session == nullptr) { std::cout << "Create session failed." << std::endl; return FAILED; } // Construct FlowGraph. Status ret = session->AddGraph(graph_id, flow_graph.ToGeGraph()); if(ret != SUCCESS) { return FAILED; } // Method 1: RunGraph ret = session->RunGraph(graph_id, input, output); if(ret != SUCCESS) { return FAILED; } // Method 2: FeedDataFlowGraph and then FetchDataFlowGraph ge::DataFlowInfo dataFlowInfo; geRet = session->FeedDataFlowGraph(0, input, dataFlowInfo, 3000); if (geRet != ge::SUCCESS) { return FAILED; } geRet = session->FetchDataFlowGraph(0, output, dataFlowInfo, 3000); if (geRet != ge::SUCCESS) { return FAILED; }

   You can import the option configuration graph to run related configuration. For details about the options, see ""Options"".

   The graph execution result will be saved to the output.

   If the DataFlow graph contains the UDF node, DataFlow saves the compilation result to the workspace directory of the specified UDF node during graph compilation. The flushed file is stored in graph_name_release.tar.gz format. This file is used for decompression in the DataFlow model deployment phase. Ensure that this file is not manually deleted during program running.

4. Call GEFinalize to destroy the allocations.

   1	ret = ge::GEFinalize();