HcclReduceScatter

Applicability

Product	Supported
Atlas 350 Accelerator Card	√
Atlas A3 training product/Atlas A3 inference product	√
Atlas A2 training product/Atlas A2 inference product	√
Atlas 200I/500 A2 inference product	☓
Atlas inference product	√
Atlas training product	√

For Atlas A2 training product/Atlas A2 inference product, only the Atlas 800T A2 training server, Atlas 900 A2 PoD cluster basic unit, and Atlas 200T A2 Box16 heterogeneous subrack are supported.

For the Atlas inference products, only the Atlas 300I Duo inference card is supported.

Function

Functions as the operation API of the ReduceScatter operator to evenly divide the input data of all ranks in a communicator into rank size parts, perform reduction (sum, prod, max, and min) on 1/rank size part of data of each rank, and distributes the result to the output buffer of each rank based on the number.

Prototype

HcclResult HcclReduceScatter(void *sendBuf, void *recvBuf, uint64_t recvCount, HcclDataType dataType, HcclReduceOp op, HcclComm comm, aclrtStream stream)

Parameters

Parameter	Input/Output	Description
sendBuf	Input	Address of the send buffer.
recvBuf	Output	Address of the buffer to receive collective communication result.
recvCount	Input	recvBuf size involved in the ReduceScatter operation. The size of sendBuf data is calculated as: recvCount × rank size.
dataType	Input	Data type of the ReduceScatter operation, which is of the HcclDataType type. Atlas 350 Accelerator Card: The supported data types are int8, int16, int32, int64, uint64, float16, float32, float64, and bfp16. Data types int64, uint64, and float64 supports only intra-node communication. Atlas A3 training product/Atlas A3 inference product: The supported data types are int8, int16, int32, int64, float16, float32, and bfp16. Atlas A2 training product/Atlas A2 inference product: The supported data types are int8, int16, int32, int64, float16, float32, and bfp16. Note that the performance will degrade for the int64 data type. Atlas training product: The supported data types are int8, int32, int64, float16, and float32. Atlas 300I Duo Inference Card: The supported data types are int8, int16, int32, float16, and float32.
op	Input	Reduction operation type. Currently, the following operation types are supported: sum, prod, max, and min. NOTE: Atlas 350 Accelerator Card: The supported operation types are sum, max, and min. Atlas A3 Training Series Product: In the current version, the prod operation does not support the int16 and bfp16 data types. Atlas A2 training product/Atlas A2 inference product: In the current version, the prod operation does not support the int16 and bfp16 data types. Atlas 300I Duo Inference Card: In the current version, the prod, max, and min operations do not support the int16 data type.
comm	Input	Communicator where the operation is performed.
stream	Input	Stream of the rank.

Returns

HcclResult: HCCL_SUCCESS on success, or else failure.

Constraints

All ranks must have the same recvCount, dataType, and op.
Atlas 300I Duo Inference Card: Only the single-server scenario is supported, and a maximum of 16 Atlas 300I Duo Inference Cards (32 NPUs) can be deployed on a single server.
The input and output addresses (sendBuf and recvBuf) of the operator must meet the following alignment requirements based on different data types:
- int8: 1-byte aligned
- int16, float16, bfp16: 2-byte aligned
- int32 and float32: 4-byte aligned
- int64, uint64, float64: 8-byte aligned

Call Example

uint32_t rankSize = 8;
uint64_t recvCount = 1;  // Number of data elements received by each rank.
uint64_t sendSize = rankSize * recvCount * sizeof(float);
uint64_t recvSize = recvCount * sizeof(float);

// Allocate device memory for collective communication.
void *sendBuf = nullptr, *recvBuf = nullptr;
aclrtMalloc(&sendBuf, sendSize, ACL_MEM_MALLOC_HUGE_ONLY);
aclrtMalloc(&recvBuf, recvSize, ACL_MEM_MALLOC_HUGE_ONLY);

// Initialize the communicator and streams.
HcclComm hcclComm;
HcclCommInitRootInfo(rankSize, &rootInfo, deviceId, &hcclComm);

// Execute ReduceScatter to sum up sendBuf of all ranks and then evenly distribute the result to the recvBuf of each rank based on the rank ID sequence.
HcclReduceScatter(sendBuf, recvBuf, recvCount, HCCL_DATA_TYPE_FP32, HCCL_REDUCE_SUM, hcclComm, stream);
// Wait until the collective communication task in the task flow is complete.
aclrtSynchronizeStream(stream);

// Free resources.
aclrtFree(sendBuf);          // Free the device memory.
aclrtFree(recvBuf);          // Free the device memory.
aclrtDestroyStream(stream);  // Destroy the task flow.
HcclCommDestroy(hcclComm);   // Destroy the communicator.

Parent topic: Collective Communication