HcclAlltoAll

Applicability

Product	Supported
Atlas 350 Accelerator Card	√
Atlas A3 training product/Atlas A3 inference product	√
Atlas A2 training product/Atlas A2 inference product	√
Atlas 200I/500 A2 inference product	☓
Atlas inference product	√
Atlas training product	√

For Atlas A2 training product/Atlas A2 inference product, only the Atlas 800T A2 training server, Atlas 900 A2 PoD cluster basic unit, and Atlas 200T A2 Box16 heterogeneous subrack are supported.

For the Atlas inference products, only the Atlas 300I Duo inference card is supported.

Function

Sends the same-sized data to all ranks in the communicator and receives the same-sized data from all ranks.

The AlltoAll operation divides input data into blocks in a specific dimension, sends the blocks to other ranks in sequence, receives input data from other ranks, and concatenates data in a specific dimension in sequence.

Prototype

HcclResult HcclAlltoAll(const void *sendBuf, uint64_t sendCount, HcclDataType sendType, const void *recvBuf, uint64_t recvCount, HcclDataType recvType, HcclComm comm, aclrtStream stream)

Parameters

Parameter	Input/Output	Description
sendBuf	Input	Address of the source data buffer.
sendCount	Input	Volume of data sent to each rank.
sendType	Input	Data type of the data to be sent, which is of the HcclDataType type. Atlas 350 Accelerator Card: The supported data types are int8, uint8, int16, uint16, int32, uint32, int64, uint64, float8-e5m2, float8-e4m3, float8-e8m0, hifloat8, float16, float32, float64, and bfp16. Atlas A3 training product/Atlas A3 inference product: The supported data types are int8, uint8, int16, uint16, int32, uint32, int64, uint64, float16, float32, float64, and bfp16. Atlas A2 training products: The supported data types are int8, uint8, int16, uint16, int32, uint32, int64, uint64, float16, float32, float64, and bfp16. Atlas training product: The supported data types are int8, uint8, int16, uint16, int32, uint32, int64, uint64, float16, float32, and float64. Atlas 300I Duo Inference Card: The supported data types are int8, uint8, int16, uint16, int32, uint32, int64, uint64, float16, float32, and float64.
recvBuf	Output	Address of the buffer to receive collective communication results. The addresses configured for recvBuf and sendBuf must be different.
recvCount	Input	Size of data received from each rank. The value must be the same as that of sendCount.
recvType	Input	Type of the received data, which is of the HcclDataType type. The value must be the same as that of sendType. Atlas 350 Accelerator Card: The supported data types are int8, uint8, int16, uint16, int32, uint32, int64, uint64, float8-e5m2, float8-e4m3, float8-e8m0, hifloat8, float16, float32, float64, and bfp16. Atlas A3 training product/Atlas A3 inference product: The supported data types are int8, uint8, int16, uint16, int32, uint32, int64, uint64, float16, float32, float64, and bfp16. Atlas A2 training product/Atlas A2 inference product: The supported data types are int8, uint8, int16, uint16, int32, uint32, int64, uint64, float16, float32, float64, and bfp16. Atlas training product: The supported data types are int8, uint8, int16, uint16, int32, uint32, int64, uint64, float16, float32, and float64. Atlas 300I Duo Inference Card: The supported data types are int8, uint8, int16, uint16, int32, uint32, int64, uint64, float16, float32, and float64.
comm	Input	Communicator where the operation is performed.
stream	Input	Stream of the rank.

Returns

HcclResult: HCCL_SUCCESS on success, or else failure.

Constraints

The ranks must have the same sendCount, sendType, recvCount, and recvType.
The performance of the AlltoAll operation is related to the size of the buffer for storing shared data between NPUs. When the communication data size exceeds the buffer size, the performance deteriorates significantly. If the AlltoAll communication data size in the service is large, you are advised to increase the buffer size appropriately by setting environment variable HCCL_BUFFSIZE to improve the communication performance.
Atlas training product: The AlltoAll communicators must meet the following requirement:
The communicators of 1p and 2p in a single server must be in the same cluster (with devices 0–3 and devices 4–7 each belonging to a separate cluster). In the communicators of 4p and 8p in a single server and multiple servers, the ranks must be based on the clusters, and the selected clusters in servers must be consistent.
Atlas training product: If a single server is used, the NIC must be in the up state. Otherwise, this API fails to be executed.
Atlas 300I Duo Inference Card: Only the single-server scenario is supported, and a maximum of 2 Atlas 300I Duo Inference Cards (4 NPUs) can be deployed on a single server.

Call Example

// Allocate device memory for collective communication.
void *sendBuf = nullptr;
void *recvBuf = nullptr;
uint64_t count = 8;
size_t mallocSize = count * sizeof(float);
aclrtMalloc((void **)&sendBuf, mallocSize, ACL_MEM_MALLOC_HUGE_ONLY);
aclrtMalloc((void **)&recvBuf, mallocSize, ACL_MEM_MALLOC_HUGE_ONLY);

// Initialize the communicator.
uint32_t rankSize = 8;
HcclComm hcclComm;
HcclCommInitRootInfo(rankSize, &rootInfo, deviceId, &hcclComm);

// Create a task flow.
aclrtStream stream;
aclrtCreateStream(&stream);

// Perform AlltoAll to send data of the same size to all ranks in the communicator and receive data of the same size from all ranks.
size_t perCount = count / rankSize;
HcclAlltoAll(sendBuf, perCount, HCCL_DATA_TYPE_FP32, recvBuf, perCount, HCCL_DATA_TYPE_FP32, hcclComm, stream);
// Wait until the collective communication task in the task flow is complete.
aclrtSynchronizeStream(stream);

// Free resources.
aclrtFree(sendBuf);          // Free the device memory.
aclrtFree(recvBuf);          // Free the device memory.
aclrtDestroyStream(stream);  // Destroy the task flow.
HcclCommDestroy(hcclComm);   // Destroy the communicator.

Parent topic: Collective Communication