GE Native Graph Construction for Graph Integration

GE native graph construction is the most direct approach for graph integration. It bypasses the framework adaptation layer and uses GE APIs directly to create operator nodes. Characteristics of GE native graph construction:

• APIs are directly called to construct a graph, with no framework conversion layer involved.
• The operator type must be the same as the registration name in REG_AUTO_MAPPING_OP.
• The REG_OP proto header file needs to be manually written.

The deliverables involved in each phase are as follows:

1. Phase 1: Operator definition

   In the operator definition phase, operator metadata, such as the inputs, outputs, and attributes, is declared for use in subsequent phases.

   Implementation by frontend framework:

   Frontend Framework	Definition Method	Deliverable	Description
   Native GE	REG_OP header file	op_graph/include/*.h	Manually written proto definition

   Example of native GE REG_OP:

   // op_graph/include/add_custom.h
   REG_OP(AddCustom)
       .INPUT(x, TensorType({DT_FLOAT, DT_FLOAT16}))
       .INPUT(y, TensorType({DT_FLOAT, DT_FLOAT16}))
       .OUTPUT(z, TensorType({DT_FLOAT, DT_FLOAT16}))
       .OP_END_FACTORY_REG(AddCustom);

2. Phase 2: Frontend integration

   There is no framework conversion layer. APIs are directly called.

3. Phase 3: Graph construction and integration

   There is no conversion mechanism. Graphs are directly constructed using APIs. The following is an example:

   // Create a custom operator node using a GE API.
   // "add_node" is the node name, and "AddCustom" is the operator type (same as REG_OP).
   auto op = ge::OperatorFactory::CreateOperator("add_node", "AddCustom");
   // Set the operator input ports.
   op.SetInput("x", input_x);  // Connect input_x to the x input port of the operator.
   op.SetInput("y", input_y);  // Connect input_y to the y input port of the operator.
   // Add the operator node to the computational graph.
   graph.AddOp(op);

4. Phase 4: Graph compilation

   The callback APIs for graph compilation are triggered during GE compilation and are used for shape inference, kernel compilation, and serialization. Callback APIs and triggering timing are described in the table below.

   Callback API	Triggering Timing	Application Scenario
   InferShape	Compile-time shape inference	All scenarios (optional)
   InferDataType	Compile-time data type inference	All scenarios (optional)
   Compile	Compile-time RTC kernel compilation	Online compilation/Offline OM
   Serialize	Writing data to the OM during ATC compilation	Offline OM

   For details about the callback API examples, see Callback APIs.

5. Phase 5: Graph execution

   This phase is the core of custom operator integration. GE calls back the Execute API during execution.

   Callback APIs and triggering timing are described in the table below.

   Callback API	Triggering Timing	Application Scenario
   Execute	Runtime kernel execution	All scenarios (mandatory)
   Deserialize	Restoring binary during OM loading	Offline OM
   UpdateHostArgs	Updates upon I/O address changes	Static graph offloading

   For details about the callback API examples, see Callback APIs.

Mapping between GE native graph integration phases and deliverables