Function Description

As the data plane entry of the cluster, the Coordinator provides the load balancing scheduling algorithm and cache affinity scheduling algorithm. As the inference request entry of users, the Coordinator selects the optimal node, monitors requests, and forwards requests based on the cluster node status and the configured scheduling algorithm, improving node resource utilization in a cluster. Figure 1 shows the architecture of the Coordinator.