Application Scenarios

DataFlow can be used in the multi-model cascading offload execution scenario. In this scenario, DataFlow can orchestrate custom compute nodes or models and execute them in a data-driven manner. Nodes defined as GraphProcessPoints can be offloaded to the device for execution, and data transfer between two adjacent GraphProcessPoint nodes is performed between devices. This mechanism reduces control-plane and data-plane interactions between the host and device, thereby lowering the execution latency of the entire DataFlow graph and improving overall performance.

As shown in the following figure, the DataFlow graph contains two FuncProcessPoints (UDF0 and UDF1). The two GraphProcessPoints are used to load the ONNX model and PB model for inference, respectively. The GraphProcessPoints can be offloaded to the device for execution. After being processed by UDF0 and UDF1, the data is transferred to the device. The execution of the two models and data transfer are completely performed on the device. For details about the sample code, see Sample Reference.