LinkLlmClusters

Applicability

Product	Supported (√/x)
Atlas 350 Accelerator Card	x
Atlas A3 training product/Atlas A3 inference product	√
Atlas A2 training product/Atlas A2 inference product	√
Atlas 200I/500 A2 inference product	x
Atlas inference product	x
Atlas training product	x

Note: For Atlas A2 training product/Atlas A2 inference product, only the Atlas 800I A2 inference server and A200I A2 Box heterogeneous subrack are supported.

Function Description

Establishes a link between two LLM-DataDist clusters. This API is called by the client.

Prototype

Status LinkLlmClusters(const std::vector<ClusterInfo> &clusters, std::vector<Status> &rets, int32_t timeout_in_millis = 1000)

Parameters

Parameter	Input/Output	Description
clusters	Input	Cluster information required for link establishment. The type is ClusterInfo. The remote_cluster_id and remote_ip_infos fields in the ClusterInfo structure must be configured. If OPTION_LOCAL_COMM_RES is not specified in Initialize, local_ip_infos must be additionally configured. For each ClusterInfo, both local_ip_infos and remote_ip_infos can contain only one IP address (ip_info) each.
rets	Output	Link establishment result of each cluster.
timeout_in_millis	Input	Link establishment timeout interval, in milliseconds. The default interval is 1000 ms.

Example

Click Gitee, download the sample package of the matching version based on the tag name, and obtain the sample from the cplusplus/level1_single_api/11_llm_data_dist directory.

Returns

LLM_SUCCESS: A success message is returned only when links are successfully established for all clusters.
Other values: Link establishment fails. Check the link establishment result of each cluster.

Exception Handling

LLM_ALREADY_LINK: A link has been established between the current cluster and the remote cluster.
LLM_LINK_FAILED: A link fails to be established.

Restrictions

Before calling this API, call the Initialize API on the client and server to complete initialization.
The maximum number of communication links that can be created is 512; creating too many links may lead to OOM errors and performance degradation during KV cache transmission.
This restriction applies to the following products:
- Atlas A2 training product/Atlas A2 inference product
- Atlas A3 training product/Atlas A3 inference product
You are advised to set the timeout interval longer than 200 ms. If TLS is enabled, it is recommended that the timeout period be set to a value greater than 2000 ms. Run the following command to query the TLS status:
```
hccn_tool [-i %d] -tls -g [host]
```
Before calling this API, register all memories on the client and server. Otherwise, memories registered after link establishment will not support remote access.
In container scenarios, if OPTION_LOCAL_COMM_RES is not configured or is set to an empty string, you must map the /etc/hccn.conf file into the container or ensure that hccn_tool exists in the default path /usr/local/Ascend/driver/tools. If neither condition is met, you need to add the directory containing hccn_tool to the PATH environment variable. A configuration example is provided below, where hccn_tool_install_path indicates the directory containing hccn_tool.
```
export PATH=$PATH:${hccn_tool_install_path}
```
This restriction applies to the following products:
- Atlas A2 training product/Atlas A2 inference product
- Atlas A3 training product/Atlas A3 inference product

Parent topic: LLM-DataDist APIs