link_clusters

Applicability

Note: For Atlas A2 training product/Atlas A2 inference product, only the Atlas 800I A2 inference server and A200I A2 Box heterogeneous subrack are supported.

Function Description

Establishes a link unilaterally. The request is initiated by the client. The server/client roles are independent of the prompt/decoder roles. Setting listen_ip_info on a node designates the node as the server (port listens for links).

Prototype

1

link_clusters(clusters: Union[List[LLMClusterInfo], Tuple[LLMClusterInfo]], timeout=3000)

Parameters

Example

Click GitCode, select the matching version, and obtain the sample from the examples/python directory.

Returns

In normal cases, a tuple of two values is returned. The first value is the return value of the API, and the type is LLMStatusCode. The second value is the list of link establishment results of each cluster, and the type is LLMStatusCode.

If a parameter is incorrect, a TypeError or ValueError may be thrown.

Restrictions

• The requirements for link establishment are as follows: Excessive links may cause OOM errors and deteriorate the transmission performance of the KV cache.
  • The maximum number of communication links that can be created is 512.
• You are advised to set the timeout interval longer than 200 ms. If TLS is enabled, it is recommended that the timeout period be set to a value greater than 2000 ms. Run the following command to query the TLS status:

  hccn_tool [-i %d] -tls -g [host]

• Register all memories before calling this API. Otherwise, memories registered after link establishment will not support remote access.
• In container scenarios, if local_comm_res is not configured or is set to an empty string, you must map the /etc/hccn.conf file into the container or ensure that hccn_tool exists in the default path /usr/local/Ascend/driver/tools. If neither condition is met, you need to add the directory containing hccn_tool to the PATH environment variable. A configuration example is provided below, where hccn_tool_install_path indicates the directory containing hccn_tool.

  export PATH=$PATH:${hccn_tool_install_path}