HcclAlltoAllV
Applicability
Product |
Supported |
|---|---|
Atlas 350 Accelerator Card |
√ |
√ |
|
√ |
|
☓ |
|
√ |
|
√ |
For
For the
Function
Sends data (with customizable size) to all ranks in the communicator and receives data from all ranks.
Prototype
1 | HcclResult HcclAlltoAllV(const void *sendBuf, const void *sendCounts, const void *sdispls, HcclDataType sendType, const void *recvBuf, const void *recvCounts, const void *rdispls, HcclDataType recvType, HcclComm comm, aclrtStream stream) |
Parameters
Parameter |
Input/Output |
Description |
|---|---|---|
sendBuf |
Input |
Address of the send buffer. |
sendCounts |
Input |
Amount of data to be sent, a uint64 array. sendCounts[i] = n indicates that the amount of data sent by the current rank to rank i is n. For example, if sendType is set to float32 and sendCounts[i] is set to n, the current rank sends n pieces of float32 data to rank i. |
sdispls |
Input |
Sending offset, a uint64 array. sdispls[i] = n indicates the offset of the start position of the data to be sent from the current rank to rank i relative to sendBuf. The basic unit is sendType. |
sendType |
Input |
Data type of the data to be sent, which is of the HcclDataType type. Atlas 350 Accelerator Card: The supported data types are int8, uint8, int16, uint16, int32, uint32, int64, uint64, float8-e5m2, float8-e4m3, float8-e8m0, hifloat8, float16, float32, float64, and bfp16. Atlas 300I Duo Inference Card: The supported data types are int8, uint8, int16, uint16, int32, uint32, int64, uint64, float16, float32, and float64. |
recvBuf |
Output |
Address of the buffer to receive collective communication results. The addresses configured for recvBuf and sendBuf must be different. |
recvCounts |
Input |
Amount of data received, a uint64 array. recvCounts[i] = n indicates that the amount of data received by the current rank from rank i is n. For example, if recvType is float32 and recvCounts[i] is n, the rank receives n pieces of float32 data from rank i. |
rdispls |
Input |
Receiving offset, a uint64 array. rdispls[i] = n indicates the offset of the start position where the data received by the current rank from rank i is stored relative to recvBuf. The basic unit is recvType. |
recvType |
Input |
Data type of the data to be received, which is of the HcclDataType type. Atlas 350 Accelerator Card: The supported data types are int8, uint8, int16, uint16, int32, uint32, int64, uint64, float8-e5m2, float8-e4m3, float8-e8m0, hifloat8, float16, float32, float64, and bfp16. Atlas 300I Duo Inference Card: The supported data types are int8, uint8, int16, uint16, int32, uint32, int64, uint64, float16, float32, and float64. |
comm |
Input |
Communicator where the operation is performed. |
stream |
Input |
Stream of the rank. |
Returns
HcclResult: HCCL_SUCCESS on success, or else failure.
Constraints
- The performance of the AlltoAllV operation is related to the size of the buffer for storing shared data between NPUs. When the communication data size exceeds the buffer size, the performance deteriorates significantly. If the AlltoAllV communication data size in the service is large, you are advised to increase the buffer size appropriately by setting environment variable HCCL_BUFFSIZE to improve the communication performance.
Atlas training product : The AlltoAllV communicators must meet the following requirement:In a cluster network, the communicators of 1p and 2p in a single server must be in the same cluster (with devices 0–3 and devices 4–7 each belonging to a separate cluster). In the communicators of 4p and 8p in a single server and multiple servers, the ranks must be based on the clusters, and the selected clusters in servers must be consistent.
Atlas training product : If a single server is used, the NIC must be in the up state. Otherwise, this API fails to be executed.- Atlas 300I Duo Inference Card: Only the single-server scenario is supported, and a maximum of 2 Atlas 300I Duo Inference Cards (4 NPUs) can be deployed on a single server.
Call Example
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 | // Allocate device memory for collective communication. void *sendBuf = nullptr; void *recvBuf = nullptr; uint64_t count = 8; size_t mallocSize = count * sizeof(float); aclrtMalloc((void **)&sendBuf, mallocSize, ACL_MEM_MALLOC_HUGE_ONLY); aclrtMalloc((void **)&recvBuf, mallocSize, ACL_MEM_MALLOC_HUGE_ONLY); // Initialize the communicator. uint32_t rankSize = 8; HcclComm hcclComm; HcclCommInitRootInfo(rankSize, &rootInfo, deviceId, &hcclComm); // Create a task flow. aclrtStream stream; aclrtCreateStream(&stream); // Set the amount of data to be sent and received. The amount of data to be sent is the same as that of data to be received. std::vector<uint64_t> sendCounts(rankSize, 1); std::vector<uint64_t> recvCounts(rankSize, 1); std::vector<uint64_t> sdispls(rankSize); std::vector<uint64_t> rdispls(rankSize); for (size_t i = 0; i < rankSize; ++i) { sdispls[i] = i; rdispls[i] = i; } // Perform AlltoAllV to send data of the same size to all ranks in the communicator and receive data of the same size from all ranks. The data size can be customized. HcclAlltoAllV(sendBuf, sendCounts.data(), sdispls.data(), HCCL_DATA_TYPE_FP32, recvBuf, recvCounts.data(), rdispls.data(), HCCL_DATA_TYPE_FP32, hcclComm, stream); // Wait until the collective communication task in the task flow is complete. aclrtSynchronizeStream(stream); // Free resources. aclrtFree(sendBuf); // Free the device memory. aclrtFree(recvBuf); // Free the device memory. aclrtDestroyStream(stream); // Destroy the task flow. HcclCommDestroy(hcclComm); // Destroy the communicator. |