Applicability

Product	Supported
Atlas 350 Accelerator Card	√
Atlas A3 training product/Atlas A3 inference product	√
Atlas A2 training product/Atlas A2 inference product	√
Atlas 200I/500 A2 inference product	☓
Atlas inference product	√
Atlas training product	√

For Atlas A2 training product/Atlas A2 inference product, only the Atlas 800T A2 training server, Atlas 900 A2 PoD cluster basic unit, and Atlas 200T A2 Box16 heterogeneous subrack are supported.

For the Atlas inference products, only the Atlas 300I Duo inference card is supported.

Function

Sends data (with customizable size) to all ranks in the communicator and receives data from all ranks.

Prototype

HcclResult HcclAlltoAllV(const void *sendBuf, const void *sendCounts, const void *sdispls, HcclDataType sendType, const void *recvBuf, const void *recvCounts, const void *rdispls, HcclDataType recvType, HcclComm comm, aclrtStream stream)

Parameters

Parameter	Input/Output	Description
sendBuf	Input	Address of the send buffer.
sendCounts	Input	Amount of data to be sent, a uint64 array. sendCounts[i] = n indicates that the amount of data sent by the current rank to rank i is n. For example, if sendType is set to float32 and sendCounts[i] is set to n, the current rank sends n pieces of float32 data to rank i.
sdispls	Input	Sending offset, a uint64 array. sdispls[i] = n indicates the offset of the start position of the data to be sent from the current rank to rank i relative to sendBuf. The basic unit is sendType.
sendType	Input	Data type of the data to be sent, which is of the HcclDataType type. Atlas 350 Accelerator Card: The supported data types are int8, uint8, int16, uint16, int32, uint32, int64, uint64, float8-e5m2, float8-e4m3, float8-e8m0, hifloat8, float16, float32, float64, and bfp16. Atlas A3 training product/Atlas A3 inference product: The supported data types are int8, uint8, int16, uint16, int32, uint32, int64, uint64, float16, float32, float64, and bfp16. Atlas A2 training product/Atlas A2 inference product: The supported data types are int8, uint8, int16, uint16, int32, uint32, int64, uint64, float16, float32, float64, and bfp16. Atlas training product: The supported data types are int8, uint8, int16, uint16, int32, uint32, int64, uint64, float16, float32, and float64. Atlas 300I Duo Inference Card: The supported data types are int8, uint8, int16, uint16, int32, uint32, int64, uint64, float16, float32, and float64.
recvBuf	Output	Address of the buffer to receive collective communication results. The addresses configured for recvBuf and sendBuf must be different.
recvCounts	Input	Amount of data received, a uint64 array. recvCounts[i] = n indicates that the amount of data received by the current rank from rank i is n. For example, if recvType is float32 and recvCounts[i] is n, the rank receives n pieces of float32 data from rank i.
rdispls	Input	Receiving offset, a uint64 array. rdispls[i] = n indicates the offset of the start position where the data received by the current rank from rank i is stored relative to recvBuf. The basic unit is recvType.
recvType	Input	Data type of the data to be received, which is of the HcclDataType type. Atlas 350 Accelerator Card: The supported data types are int8, uint8, int16, uint16, int32, uint32, int64, uint64, float8-e5m2, float8-e4m3, float8-e8m0, hifloat8, float16, float32, float64, and bfp16. Atlas A3 training product/Atlas A3 inference product: The supported data types are int8, uint8, int16, uint16, int32, uint32, int64, uint64, float16, float32, float64, and bfp16. Atlas A2 training product/Atlas A2 inference product: The supported data types are int8, uint8, int16, uint16, int32, uint32, int64, uint64, float16, float32, float64, and bfp16. Atlas training product: The supported data types are int8, uint8, int16, uint16, int32, uint32, int64, uint64, float16, float32, and float64. Atlas 300I Duo Inference Card: The supported data types are int8, uint8, int16, uint16, int32, uint32, int64, uint64, float16, float32, and float64.
comm	Input	Communicator where the operation is performed.
stream	Input	Stream of the rank.

Returns

HcclResult: HCCL_SUCCESS on success, or else failure.

Constraints

The performance of the AlltoAllV operation is related to the size of the buffer for storing shared data between NPUs. When the communication data size exceeds the buffer size, the performance deteriorates significantly. If the AlltoAllV communication data size in the service is large, you are advised to increase the buffer size appropriately by setting environment variable HCCL_BUFFSIZE to improve the communication performance.

Atlas training product: The AlltoAllV communicators must meet the following requirement:
In a cluster network, the communicators of 1p and 2p in a single server must be in the same cluster (with devices 0–3 and devices 4–7 each belonging to a separate cluster). In the communicators of 4p and 8p in a single server and multiple servers, the ranks must be based on the clusters, and the selected clusters in servers must be consistent.
Atlas training product: If a single server is used, the NIC must be in the up state. Otherwise, this API fails to be executed.
Atlas 300I Duo Inference Card: Only the single-server scenario is supported, and a maximum of 2 Atlas 300I Duo Inference Cards (4 NPUs) can be deployed on a single server.

Call Example

// Allocate device memory for collective communication.
void *sendBuf = nullptr;
void *recvBuf = nullptr;
uint64_t count = 8;
size_t mallocSize = count * sizeof(float);
aclrtMalloc((void **)&sendBuf, mallocSize, ACL_MEM_MALLOC_HUGE_ONLY);
aclrtMalloc((void **)&recvBuf, mallocSize, ACL_MEM_MALLOC_HUGE_ONLY);

// Initialize the communicator.
uint32_t rankSize = 8;
HcclComm hcclComm;
HcclCommInitRootInfo(rankSize, &rootInfo, deviceId, &hcclComm);

// Create a task flow.
aclrtStream stream;
aclrtCreateStream(&stream);

// Set the amount of data to be sent and received. The amount of data to be sent is the same as that of data to be received.
std::vector<uint64_t> sendCounts(rankSize, 1);
std::vector<uint64_t> recvCounts(rankSize, 1);
std::vector<uint64_t> sdispls(rankSize);
std::vector<uint64_t> rdispls(rankSize);
for (size_t i = 0; i < rankSize; ++i) {
    sdispls[i] = i;
    rdispls[i] = i;
}
// Perform AlltoAllV to send data of the same size to all ranks in the communicator and receive data of the same size from all ranks. The data size can be customized.
HcclAlltoAllV(sendBuf, sendCounts.data(), sdispls.data(), HCCL_DATA_TYPE_FP32,
              recvBuf, recvCounts.data(), rdispls.data(), HCCL_DATA_TYPE_FP32, hcclComm, stream);
// Wait until the collective communication task in the task flow is complete.
aclrtSynchronizeStream(stream);

// Free resources.
aclrtFree(sendBuf);          // Free the device memory.
aclrtFree(recvBuf);          // Free the device memory.
aclrtDestroyStream(stream);  // Destroy the task flow.
HcclCommDestroy(hcclComm);   // Destroy the communicator.