RunGraphAsync

Applicability

Product	Supported
Atlas 350 Accelerator Card	√
Atlas A3 training product/Atlas A3 inference product	√
Atlas A2 training product/Atlas A2 inference product	√
Atlas 200I/500 A2 inference product	x
Atlas inference product	√
Atlas training product	√

Header File/Library File

Header file: #include <ge/ge_api_v2.h>
Library file: libge_runner_v2.so

Function Usage

Asynchronously runs the graph of a specified ID and returns the execution result.

This API is mutually exclusive with RunGraph/RunGraphWithStreamAsync. If a graph has not been loaded using LoadGraph before this API is called, this API automatically calls LoadGraph to complete the loading.
Both this API and RunGraph are used to run the graph of a specified ID and output the result. Specifically:
- This API is asynchronous.
- You can customize the RunAsyncCallbackV2 callback function to obtain the computation result of the graph. When Status of the callback function is SUCCESS, data can be processed.

Prototype

Status RunGraphAsync(uint32_t graph_id, const std::vector<gert::Tensor> &inputs, RunAsyncCallbackV2 callback)

Parameters

Parameter

Input/Output

Description

graph_id

Input

Subgraph ID.

inputs

Input

Input data of the computational graph, which can be located on a host or device.

If the data is stored on the host but is executed on the device, when user inputs enter a data queue, memory must be allocated and copied for each input. A large input size or quantity may result in a performance bottleneck.

callback

Output

Callback function of the current subgraph. The output is the host memory allocated by GE.

using Status = uint32_t;
using RunAsyncCallbackV2 = std::function<void(Status, std::vector<gert::Tensor> &)>;

Returns

Parameter	Type	Description
-	Status	GE_CLI_GE_NOT_INITIALIZED: GE is not initialized. SUCCESS: The graph is successfully run by using the asynchronous API. FAILED: The graph fails to be run by using the asynchronous API.

Parameter

Type

Description

Status

GE_CLI_GE_NOT_INITIALIZED: GE is not initialized.

SUCCESS: The graph is successfully run by using the asynchronous API.

FAILED: The graph fails to be run by using the asynchronous API.

Restrictions

If the input tensor data is stored on the device, the storage address of the tensor on the device must be 32-byte aligned. Otherwise, an undefined error may occur.

Example

Call AddGraph to load a subgraph.

std::map <AscendString, AscendString> options;
ge::GeSession *session = new GeSession(options);

uint32_t graph_id = 0;
ge::Graph graph;
Status ret = session->AddGraph(graph_id, graph);

Customize RunAsyncCallbackV2 to process data as follows:

void CallBack(Status result, std::vector<gert::Tensor> &out_tensor) {
if(result == ge::SUCCESS) {
// Read the out_tensor data and process the data as required.
for(auto &tensor : out_tensor) {
  auto data = tensor.GetAddr();
  int64_t length = tensor.GetSize();
  }
}
}

Define the input data of the graph: const std::vector<gert::Tensor> &inputs
Call RunGraphAsync(graph_id, inputs, CallBack).

Parent topic: GESession