DataFlow Framework

DataFlow Framework Structure

Figure 1 DataFlow framework structure

The DataFlow consists of three layers, as described in the following:

• DataFlow API: defines heterogeneous compute flows. It is used by the app developers to describe service data flows and generate computational graphs based on Graph IR. For details, see Data Flow Representation.
• Heterogeneous parallel compiler: compiles heterogeneous computational graphs. Based on the composition and connection topology of heterogeneous compute units, it performs graph partitioning, graph optimization, and graph deployment, and generates executable binaries for each heterogeneous compute unit.
• Heterogeneous runtime: schedules executable binaries to corresponding heterogeneous execution units according to outputs generated by the heterogeneous parallel compiler. For details, see Heterogeneous Runtime.

Data Flow Representation

• The rules and constraints of the data flow diagram (DFD) are as follows:
  • A FlowGraph is a DAG. The data flows are directed and cyclic expressions are not allowed.
  • FlowGraphs without input or output are not supported.
  • Data transferred between nodes may be shared by multiple nodes. To ensure immutability, compute nodes are not allowed to modify input data. If necessary, copy operations shall be performed.
  • One-to-many data flows are supported, that is, the same data is copied and distributed to multiple nodes. However, multiple outputs of the same node are sent to the same input.

    

  • The representation can be executed by calling GraphProcessPoint to implement control flow representations, such as loop iterations. The following figure shows the branch control logic.

    

Heterogeneous Runtime

Heterogeneous runtime provides different executors for heterogeneous CPU and NPU nodes, including:

• CPU graph executor: deployed on CPU nodes to execute computational graphs.
• NPU graph executor: deployed on NPU nodes to execute computational graphs.
• UDF executor: execution engine for custom functions, which can be flexibly deployed on CPU nodes or AI CPUs on NPUs.

All executors are deployed as independent processes. Inter-process data communication adopts the following mechanisms: shared memory and shared queues for intra-node transmission, and high-performance RDMA/UB communication for inter-node transmission.

The data flow gateway (FlowGW) proxies and forwards the data flow production and consumption relationship between executors, one-to-many distribution relationship, and cross-node data communication.