文档

Graph Build Procedure

This section describes several typical service scenarios of graph build and the interaction process in each scenario.

A network model is usually a graph composed of tensors and nodes (or operators). Notes:

  • A tensor consists of its data and description, which describes its name, dtype, shape, and format.
  • An operator consists of its name, type, input, and attributes.
  • A graph consists of the network name, operator list, and input and output operators.

There are three graph build scenarios. Select one based on your requirements.

Table 1 Graph build and run

Scenario

Description

Chip

Graph Build to an Offline Model

Build a graph and save it as an OM offline model. The AscendCL API loads the offline model and performs model inference.

For details, see section "Model Inference" in CANN AscendCL Application Software Development Guide (C&C++).

All

Compiling and Running a Graph

After building a graph, compile and run the graph and use the API in AddGraph to load the graph object.

  • Synchronous mode: After the graph is loaded, use RunGraph to run the graph synchronously to obtain the result.
  • Asynchronous mode: After the graph is loaded, use RunGraphWithStreamAsync to run the graph asynchronously to obtain the result.

Atlas Training Series Product

Compiling a Graph to an Offline Model and Running the Graph (Distributed Compilation and Partitioning of Foundation Models)

Applicable to the distributed compilation and partition of foundation models.

Compile an OM offline model that can be used for distributed deployment, use LoadGraph to load the model, and use RunGraph to run the graph that loads the model to obtain the graph execution result.

Atlas Training Series Product

Graph Build to an Offline Model

In this scenario, a graph is compiled and saved as an OM offline model. Figure 1 shows the process. For details about services, see Modifying a Graph Using a Custom Pass.

Figure 1 Model build process

Compiling and Running a Graph

Figure 2 shows the workflow of compiling and running a graph. For details about services, see Graph Build to an .om Offline Model.

Figure 2 Graph compilation and running

Compiling a Graph to an Offline Model and Running the Graph (Distributed Compilation and Partitioning of Foundation Models)

Figure 3 shows the process. For details about services, see Graph Build to an Offline Model and Graph Run (Distributed Build and Partitioning of Foundation Models).

Figure 3 Graph running workflow
Parent topic: Introduction