Environment Setup

The following products are supported:

Atlas A2 training product/Atlas A2 inference product: For Atlas A2 training product/Atlas A2 inference product, only the Atlas 800I A2 inference server and A200I A2 Box heterogeneous subrack are supported. When the HCCS transmission protocol is used in the server, LLM-DataDist APIs support only D2D transmission.

Atlas A3 training product/Atlas A3 inference product: When the HCCS transmission protocol is used, LLM-DataDist APIs do not support the host memory as the remote cache.

Install the driver, firmware, and CANN by following CANN Software Installation.

Use the hccn_tool command to query device IP addresses and perform inter-device network checks. Ensure that RDMA links exist between devices in each cluster; otherwise, the LLM-DataDist capabilities cannot be enabled. For details about hccn_tool, see HCCN Tool API Reference. The following table lists the common commands.

Command	Scenario
hccn_tool [-i %d] -link -g	Obtains the link status of a specified device. -i specifies the device. Example: hccn_tool -i 0 -link -g
hccn_tool [-i %d] -ip -g	Obtains the IP address and subnet mask of a specified device. -i specifies the device. Example: hccn_tool -i 0 -ip -g
hccn_tool [-i %d] -ping -g [address %s ]	Obtains the ping result from a specified device to a destination address. -i specifies the device on the current server, and address specifies the destination address. Example: hccn_tool -i 0 -ping -g address 192.168.2.1

Command

Scenario

hccn_tool [-i %d] -link -g

Obtains the link status of a specified device. -i specifies the device.

Example:

hccn_tool -i 0 -link -g

hccn_tool [-i %d] -ip -g

Obtains the IP address and subnet mask of a specified device. -i specifies the device.

Example:

hccn_tool -i 0 -ip -g

hccn_tool [-i %d] -ping -g [address %s ]

Obtains the ping result from a specified device to a destination address. -i specifies the device on the current server, and address specifies the destination address.

Example:

hccn_tool -i 0 -ping -g address 192.168.2.1

The following environment variables are involved when LLM-DataDist is used. For details, see Environment Variables.

Environment Variable	Scenario
HCCL_RDMA_TC and HCCL_RDMA_SL	When customers have their own parameter-plane network plan and assign types and priorities to various traffic flows, use these variables to set the traffic type and priority for parameter-plane collective communication, to align with the customer's network traffic plan.
HCCL_RDMA_RETRY_CNT and HCCL_RDMA_TIMEOUT	These variables correspond to the retry count and retransmission timeout factor (timeout) of the RDMA NIC, respectively. Setting them too high makes the system insensitive to network anomalies, failing to detect faults; setting them too low may cause transient disconnections that directly interrupt services, as the NIC hardware cannot mask them. You can choose appropriate values based on your network conditions. For example, you can specify the time range based on most intermittent disconnections. You are advised to use the following formula to reduce the impact of network jitter: HCCL_RDMA_TIMEOUT=log2(pull kv timeout x 10^6/(HCCL_RDMA_RETRY_CNT + 1)/4.096), rounded up. If the pull kv timeout and HCCL_RDMA_RETRY_CNT are both set to the default values, you are advised to set HCCL_RDMA_TIMEOUT to 15.
HCCL_INTRA_ROCE_ENABLE	This variable specifies whether to use the RoCE loopback for multi-card communication on a server.

Environment Variable

Scenario

HCCL_RDMA_TC and HCCL_RDMA_SL

When customers have their own parameter-plane network plan and assign types and priorities to various traffic flows, use these variables to set the traffic type and priority for parameter-plane collective communication, to align with the customer's network traffic plan.

HCCL_RDMA_RETRY_CNT and HCCL_RDMA_TIMEOUT

These variables correspond to the retry count and retransmission timeout factor (timeout) of the RDMA NIC, respectively. Setting them too high makes the system insensitive to network anomalies, failing to detect faults; setting them too low may cause transient disconnections that directly interrupt services, as the NIC hardware cannot mask them. You can choose appropriate values based on your network conditions. For example, you can specify the time range based on most intermittent disconnections.

You are advised to use the following formula to reduce the impact of network jitter:

HCCL_RDMA_TIMEOUT=log2(pull kv timeout x 10^6/(HCCL_RDMA_RETRY_CNT + 1)/4.096), rounded up.

If the pull kv timeout and HCCL_RDMA_RETRY_CNT are both set to the default values, you are advised to set HCCL_RDMA_TIMEOUT to 15.

HCCL_INTRA_ROCE_ENABLE

This variable specifies whether to use the RoCE loopback for multi-card communication on a server.

Parent topic: Quick Start