Sample Reference

sample_multiple_model

Sample obtaining

Cascades multiple models and offloads them for execution. Constructs a DataFlow graph using FuncProcessPoint and GraphProcessPoint, and offloads the ONNX and PB models executed by GraphProcessPoint to the device to reduce the interaction between the control plane and data plane of the host and device and improve the execution performance.

sample_pytorch

Sample obtaining

sample_pytorch: Performs online model inference by PyTorch-based DataFlow. The original serial execution process of preprocessing, model inference, and postprocessing is reconstructed into a FlowGraph that can be executed asynchronously in a pipeline, improving the overall inference throughput.

sample_npu_model

Sample obtaining

Adds data offload or model offload for PyTorch model nodes for optimization of sample_pytorch.