Link Disconnection

Function Description

You can call UnlinkLlmClusters to disconnect and clean up the communication links between Prefill and Decode nodes.

Application Scenarios

When a node in the Prefill or Decode cluster is abnormal, you can clean up the abnormal link by disconnecting it. If the ratio of Prefill and Decode nodes in the cluster needs to be adjusted, disconnect the established links.

Example

There are two disconnection methods:

Initiate a disconnection request on the Decode side, which is commonly used in non-faulty link scenarios.

// clusters must be the same as those used for link establishment.
auto ret = llm_data_dist.UnLinkLlmClusters(clusters, rets);
if (ret != LLM_SUCCESS) {
    printf("[ERROR] UnLinkLlmClusters failed, ret = %u\n", ret);
    return -1;
}
for (const auto &inner_ret : rets) {
    if (inner_ret != LLM_SUCCESS) {
        printf("[ERROR] UnLinkLlmClusters failed, ret = %u\n", inner_ret);
        return -1;
    }
}

Force link disconnection from both the Prefill and Decode sides, which is typically used in link fault scenarios.

// Prefill and Decode sides
std::vector<Status> rets;
std::vector<ClusterInfo> clusters;
ClusterInfo cluster_info;
IpInfo local_ip_info;
// Replace it with the actual local IP address.
local_ip_info.ip = "ip";
IpInfo remote_ip_info;
// Replace it with the actual peer IP address.
remote_ip_info.ip = "ip";
// Replace it with the port for listening used during initialization on the Prefill side.
remote_ip_info.port = "port";
cluster_info.remote_cluster_id = PROMPT_CLUSTER_ID;
cluster_info.local_ip_infos.emplace_back(std::move(local_ip_info)); // local_ip_infos stores the local device IP addresses.
cluster_info.remote_ip_infos.emplace_back(std::move(remote_ip_info)); // remote_ip_infos stores the peer device IP addresses.
clusters.emplace_back(std::move(cluster_info));
auto ret = llmDataDist.UnLinkLlmClusters(clusters, rets, 1000, true);
if (ret != LLM_SUCCESS) {
    printf("[ERROR] UnLinkLlmClusters failed, ret = %u\n", ret);
    return -1;
}
for (const auto &inner_ret : rets) {
    if (inner_ret != LLM_SUCCESS) {
        printf("[ERROR] UnLinkLlmClusters failed, ret = %u\n", inner_ret);
        return -1;
    }
}

Troubleshooting

If UnLinkLlmClusters fails to be called due to a network fault, forcibly disconnect the link.

For more troubleshooting methods, see Error Codes.

Parent topic: Link Management