Communicator Management

A communicator is a context for executing collective communication operators. It manages corresponding communication objects (for example, NPUs) and resources required for communication. A rank is a communication object in the communicator. Each rank is assigned a unique identifier ranging from 0 to n – 1 (n is the number of NPUs).

A communicator can be created in the following ways based on user scenarios:

Multi-server collective communication
- If there is a complete rank table file that describes the cluster information, call the HcclCommInitClusterInfo API to create a communicator. For specific configurations, call HcclCommInitClusterInfoConfig.
- If there is no complete rank table file, call the HcclGetRootInfo API alongside HcclCommInitRootInfo/HcclCommInitRootInfoConfig to create a communicator using the root rank information.
For single-server collective communication, call the HcclCommInitAll API to create communicators in batches on a single server.
Call the HcclCreateSubCommConfig API to split an existing communicator into sub-communicators with specific configurations.

All communication operators in multiple communicators must be dispatched in serial mode on each device. Out-of-order dispatch, multi-thread concurrent dispatch, and thread reentry are not allowed.
On the same device, the dispatch threads of all communication operators in the same communicator must use the same context.
Graph-mode communication and single-operator communication cannot be performed together in the same communicator.
Operators in the same communicator must be executed in serial mode.
Multiple communicators need to be created in serial mode on the same NPU.
For Atlas A3 training product/Atlas A3 inference product, if there are multiple SuperPoDs on the network during communicator initialization, configure AI servers that belong to the same SuperPoD together. Assume that there are two SuperPoDs whose IDs are 0 and 1. Configure AI servers on SuperPoD 0, and then configure AI servers on SuperPoD 1. Cross-configuring AI servers on SuperPoDs 0 and 1 is not supported.

Creating a Communicator Based on the Rank Table

In the multi-server collective communication scenario, the process for creating a communicator based on the cluster information configuration file (rank table file) is as follows. Each device requires its own process.

Construct a rank table file. For details about how to configure the rank table file, see Cluster Configuration.
Each device calls the HcclCommInitClusterInfo API to create a communicator, or calls the HcclCommInitClusterInfoConfig API to create a communicator with specific configurations.

A simple code snippet is as follows:

    int devId = 0;
    // Configure the path of the rank table file.
    char* rankTableFile = "/home/rank_table.json";
    // Define the communicator handle.
    HcclComm hcclComm;
    // Initialize the HCCL communicator.
    HcclCommInitClusterInfo(rankTableFile, devId, &hcclComm);
   
    /*  Collective communication   */

    // Destroy the HCCL communicator.
    HcclCommDestroy(hcclComm);

For the Atlas 350 Accelerator Card,Atlas A3 training product/Atlas A3 inference product,Atlas A2 training product/Atlas A2 inference product, if the service runs on a single device with multiple processes, you are advised to configure the device_port field in the rank table file. Configure different port numbers for each process to avoid conflicts that can stop service running. Note that multiple processes increase resource overheads and affect communication performance.

Creating a Communicator Based on Root Rank Information

In the multi-server collective communication scenario, if there is no complete cluster configuration file (rank table file), HCCL provides the mode of creating a communicator based on root rank information. There are two typical use scenarios:

If each device corresponds to a service process, the implementation process is as follows:

For the Atlas 350 Accelerator Card, check whether the rootinfo file exists. For other products, skip this step.
Before creating a communicator based on root rank information, check whether the /etc/hccl_rootInfo.json file exists. This file stores the entity ID (EID) for NPU communication and is automatically generated after the environment is deployed. If the file does not exist, click here to contact technical support..

(Optional) Specify the communication IP address or NIC used by the host rank during HCCL initialization.

Method 1: On each host rank, set an IP address using HCCL_IF_IP for its communication with the root rank. The IP address can be in IPv4 or IPv6 format. Only one IP address can be configured. A configuration example is as follows:
1
export HCCL_IF_IP=10.10.10.1

Method 2: On each host rank, set a NIC name using HCCL_SOCKET_IFNAME and the communication protocol for the NIC using HCCL_SOCKET_FAMILY. HCCL will use the NIC name to obtain the host IP address for communication with the root rank. A configuration example is as follows:

# IP version used by the communication NIC during HCCL initialization. AF_INET indicates that IPv4 is used. AF_INET6 indicates that IPv6 is used.
export HCCL_SOCKET_FAMILY=AF_INET

# The following formats of NIC names are supported. (Select one from the four formats. If multiple NICs are configured via the environment variables, separate them with commas (,). The first matched NIC is used as the communication NIC.)
# Exact match of the NIC
export HCCL_SOCKET_IFNAME==eth0,enp0   # Use NIC eth0 or enp0.
export HCCL_SOCKET_IFNAME=^=eth0,enp0     # Do not use NIC eth0 or enp0.

# Fuzzy match of the NIC
export HCCL_SOCKET_IFNAME=eth,enp       # Use all NICs prefixed with eth or enp.
export HCCL_SOCKET_IFNAME=^eth,enp      # Do not use any NIC prefixed with eth or enp.

The priority of HCCL_IF_IP is higher than that of HCCL_SOCKET_IFNAME. If HCCL_IF_IP or HCCL_SOCKET_IFNAME is not set, the system automatically selects NICs according to the following priorities. If the NIC specified for the current rank is disconnected from that for the root rank, the HCCL link establishment will fail.

NICs other than Docker or local NICs (in ascending alphabetical order of NIC names) > Docker NICs > local NICs

Call the HcclGetRootInfo API on the root rank to generate root rank information (rootInfo), including the device IP address and device ID.
Broadcast the root rank information to all ranks in the communicator.
Call HcclCommInitRootInfo or HcclCommInitRootInfoConfig (create communicators with specific configurations) on each rank in a communicator to initialize the communicator based on the received rootInfo and the rank ID.

Each AI server corresponds to a service process, and each thread corresponds to a device. The following shows how to create a multi-device communicator using multiple threads.
1. For the Atlas 350 Accelerator Card, check whether the rootinfo file exists. For other products, skip this step.
  Before creating a communicator based on root rank information, check whether the /etc/hccl_rootInfo.json file exists. This file stores the entity ID (EID) for NPU communication and is automatically generated after the environment is deployed. If the file does not exist, click here to contact technical support..
2. (Optional) Go to 2 to specify the communication IP address or NIC used by the host rank during HCCL initialization.
3. In the main process, loop through specifying different devices and calling the HcclGetRootInfo API to obtain multiple rootInfo records.
4. Each device matches a thread. The HcclCommInitRootInfo or HcclCommInitRootInfoConfig API is called concurrently based on different rootInfo to initialize the communicator.

For the Atlas 350 Accelerator CardAtlas A3 training product/Atlas A3 inference productAtlas A2 training product/Atlas A2 inference product, if the service runs on a single device with multiple processes, you are advised to use HCCL_HOST_SOCKET_PORT_RANGE and HCCL_NPU_SOCKET_PORT_RANGE to configure the communication ports for HCCL on the host and NPU, respectively. Otherwise, port conflicts may occur. The following is a configuration example. Note that multiple processes increase resource overheads and affect communication performance.

export HCCL_HOST_SOCKET_PORT_RANGE="auto"
export HCCL_NPU_SOCKET_PORT_RANGE="auto"

Creating Communicators in Batches on a Single Server

For the single-server communication, you can use a process to create communicators for multiple devices. One device corresponds to one thread. The creation process is as follows:

Construct a device list in the communicator, for example, {0, 1, 2, 3, 4, 5, 6, 7}. The device ID in the list is the logical ID (which can be queried by running the npu-smi info -m command). The HCCL creates communicators based on the sequence set in the list.
Call the HcclCommInitAll API in the process to create communicators.

    uint32_t ndev = 8;
    // Construct the logical ID list of the devices.
    int32_t devices[8] = {0, 1, 2, 3, 4, 5, 6, 7};
    // Define the communicator handle.
    HcclComm comms[ndev];
    // Initialize the HCCL communicator.
    HcclCommInitAll(ndev, devices, comms);

    // Start the thread to perform the collective operations.
    std::vector<std::unique_ptr<std::thread> > threads(ndev);
    struct ThreadContext args[ndev];
    for (uint32_t i = 0; i < ndev; i++) {
        args[i].device = i;
        args[i].comm = comms[i];
       /*  Collective communication   */      
    }

    // Destroy the HCCL communicator.
    for (uint32_t i = 0; i < ndev; i++) {
        HcclCommDestroy(comms[i]);
    }

Note: If multiple threads call a collective communication operation API (for example, HcclAllReduce), ensure that the time gap between two calls of the API does not exceed the link setup time. To avoid link setup timeout, you can set the environment variable HCCL_CONNECT_TIMEOUT, which is 120s by default.

Splitting an Existing Communicator into Sub-Communicators

HCCL provides the HcclCreateSubCommConfig API to split an existing communicator into sub-communicators with specific configurations. In this way, a sub-communicator can be quickly created without setting up a socket link or exchanging rank information, helping fast service fault recovery.

// Initialize the global communicator.
HcclComm globalHcclComm;
HcclCommInitClusterInfo(rankTableFile, devId, &globalHcclComm);
// Configure the communicator.
HcclCommConfig config;
HcclCommConfigInit(&config);
config.hcclBufferSize = 50;
strcpy(config.hcclCommName, "comm_1");
// Initialize the sub-communicators.
HcclComm hcclComm;
uint32_t rankIds[4] = {0, 1, 2, 3};  // Rank list of the sub-communicators.
HcclCreateSubCommConfig(&globalHcclComm, 4, rankIds, 1, devId, &config, &hcclComm);

This API does not support nested splitting of communicators, that is, sub-communicators cannot be further split.

Destroying a Communicator

After a collective communication operation is complete, call the HcclCommDestroy API to destroy the communicator, and call the runtime management API to free the memory, stream, and device resources used for the communication.

Parent topic: Communication Function Implementation Using Communication Library APIs