Environment Setup

The following products are supported:

Atlas A2 training products/Atlas A2 inference products: For Atlas A2 training products/Atlas A2 inference products, only the Atlas 800I A2 inference server and A200I A2 Box heterogeneous subrack are supported.

Atlas A3 training products/Atlas A3 inference products
Ascend 950PR/Ascend 950DT: The UB protocol is used within a SuperPoD, and the RoCE protocol is used between SuperPoDs.

Install the driver, firmware, and CANN by following operations in CANN Software Installation.

Use the hccn_tool command to query device IP addresses and perform inter-device network checks. Ensure that RDMA links exist between devices in each cluster; otherwise, the HIXL capabilities cannot be enabled. For details about hccn_tool, see HCCN Tool API Reference.

The following environment variables are involved when HIXL is used. For details, see Environment Variables.

Environment Variable	Scenario
HCCL_RDMA_TC and HCCL_RDMA_SL	When customers have their own parameter-plane network plan and assign types and priorities to various traffic flows, use these variables to set the traffic type and priority for parameter-plane collective communication, to align with the customer's network traffic plan.
HCCL_RDMA_RETRY_CNT and HCCL_RDMA_TIMEOUT	These variables correspond to the retry count and retransmission timeout factor (timeout) of the RDMA NIC, respectively. Setting them too high makes the system insensitive to network anomalies, failing to detect faults; setting them too low may cause transient disconnections that directly interrupt services, as the NIC hardware cannot mask them. You can choose appropriate values based on your network conditions. For example, you can specify the time range based on most intermittent disconnections. You are advised to use the following formula to reduce the impact of network jitter: HCCL_RDMA_TIMEOUT=log2(pull kv timeout x 10^6/(HCCL_RDMA_RETRY_CNT + 1)/4.096), rounded up. NOTE: If the pull kv timeout and HCCL_RDMA_RETRY_CNT are both set to the default values, you are advised to set HCCL_RDMA_TIMEOUT to 15.
HCCL_INTRA_ROCE_ENABLE	This variable specifies whether to use the RoCE loopback for intra-server multi-device communication. When RoCE is used (that is, HCCL_INTRA_ROCE_ENABLE is set to 1), HCCL_INTRA_PCIE_ENABLE must be set to 0.

Environment Variable

Scenario

HCCL_RDMA_TC and HCCL_RDMA_SL

When customers have their own parameter-plane network plan and assign types and priorities to various traffic flows, use these variables to set the traffic type and priority for parameter-plane collective communication, to align with the customer's network traffic plan.

HCCL_RDMA_RETRY_CNT and HCCL_RDMA_TIMEOUT

These variables correspond to the retry count and retransmission timeout factor (timeout) of the RDMA NIC, respectively. Setting them too high makes the system insensitive to network anomalies, failing to detect faults; setting them too low may cause transient disconnections that directly interrupt services, as the NIC hardware cannot mask them. You can choose appropriate values based on your network conditions. For example, you can specify the time range based on most intermittent disconnections.

You are advised to use the following formula to reduce the impact of network jitter:

HCCL_RDMA_TIMEOUT=log2(pull kv timeout x 10^6/(HCCL_RDMA_RETRY_CNT + 1)/4.096), rounded up.

NOTE:

If the pull kv timeout and HCCL_RDMA_RETRY_CNT are both set to the default values, you are advised to set HCCL_RDMA_TIMEOUT to 15.

HCCL_INTRA_ROCE_ENABLE

This variable specifies whether to use the RoCE loopback for intra-server multi-device communication. When RoCE is used (that is, HCCL_INTRA_ROCE_ENABLE is set to 1), HCCL_INTRA_PCIE_ENABLE must be set to 0.

Parent topic: Quick Start