HcommLocalReduceOnThread

Applicability

Product	Supported
Atlas 350 Accelerator Card	√
Atlas A3 training product/Atlas A3 inference product	√
Atlas A2 training product/Atlas A2 inference product	√
Atlas 200I/500 A2 inference product	x
Atlas inference product	x
Atlas training product	x

For Atlas A2 training product/Atlas A2 inference product, only the Atlas 800T A2 training server, Atlas 900 A2 PoD cluster basic unit, and Atlas 200T A2 Box16 heterogeneous subrack are supported.

Function

Performs a reduceOp operation on the memory data of length count × sizeof(dataType) pointed to by src and the memory data of the same length pointed to by dst, and outputs the result to dst.

Prototype

int32_t HcommLocalReduceOnThread(ThreadHandle thread, void *dst, const void *src, uint64_t count, HcommDataType dataType, HcommReduceOp reduceOp)

Parameters

Parameter	Input/Output	Description
thread	Input	Communication thread handle, which is threads returned by the HcclThreadAcquire call. For details, see ThreadHandle.
dst	Output	Destination address, which is the device memory.
src	Input	Source address, which is the device memory.
count	Input	Number of elements.
dataType	Input	Data type. For details, see HcommDataType. Atlas 350 Accelerator Card: The supported data types are int8, int16, int32, float16, float32, and bfp16. Atlas A3 training product/Atlas A3 inference product: The supported data types are int8, int16, int32, float16, float32, and bfp16. Atlas A2 training product/Atlas A2 inference product: The supported data types are int8, int16, int32, float16, float32, and bfp16.
reduceOp	Input	Reduction operation type. For details, see HcommReduceOp. Atlas 350 Accelerator Card: The supported reduction types are sum, max, and min. Atlas A3 training product/Atlas A3 inference product: The supported reduction types are sum, max, and min. Atlas A2 training product/Atlas A2 inference product: The supported reduction types are sum, max, and min.

Returns

int32_t: 0 on success; else, failure.

Restrictions

Both dst and src must be device memory.

For the Atlas 350 Accelerator Card, this API can be called only on the device in AICPU_TS mode.

Example

HcclComm comm;
CommEngine engine = COMM_ENGINE_CPU_TS;
aclrtStream stream;
aclrtCreateStream(&stream);
ThreadHandle thread;
HcclResult result = HcclThreadAcquireWithStream(comm, engine, stream, 2, &thread);

// Allocate device memory.
uint64_t memSize = 256;
s32 policy = static_cast<int>(ACL_MEM_TYPE_HIGH_BAND_WIDTH) | static_cast<int>(ACL_MEM_MALLOC_HUGE_FIRST);
aclrtMallocAttrValue moduleIdValue;
moduleIdValue.moduleId = HCCL;
aclrtMallocAttribute attrs{.attr = ACL_RT_MEM_ATTR_MODULE_ID, .value = moduleIdValue};
aclrtMallocConfig cfg{.attrs = &attrs, .numAttrs = 1};

void* inputMem;
void* outputMem;
aclrtMallocWithCfg(&inputMem, memSize, static_cast<aclrtMemMallocPolicy>(policy), &cfg);
aclrtMallocWithCfg(&outputMem, memSize, static_cast<aclrtMemMallocPolicy>(policy), &cfg);
// Perform the reduction operation on the device.
uint64_t count = memSize / SIZE_TABLE[HCCL_DATA_TYPE_FP32];
HcommLocalReduceOnThread(thread, outputMem, inputMem, count, HCCL_DATA_TYPE_FP32, HCCL_REDUCE_SUM);

On the Atlas 350 Accelerator Card, this function needs to be compiled and used on the device.

HcclComm comm;
CommEngine engine = COMM_ENGINE_AICPU_TS;

void *cclBufferAddr = nullptr;
uint64_t cclBufferSize = 0;
HcclGetHcclBuffer(comm, &cclBufferAddr, &cclBufferSize);
ThreadHandle thread;
HcclThreadAcquire(comm, engine, 1, 1, &thread);

// Allocate other resources.
// Copy parameters and launch the kernel.

// Orchestrate algorithms on the device.
uint64_t len = 256;
void *src = param.userIn;
void *dst = param.cclBuf;
uint64_t sizeOfFP32 = 4;
uint64_t count = len / sizeOfFP32;
HcommLocalReduceOnThread(thread, dst, src, count, HCOMM_DATA_TYPE_FP32, HCOMM_REDUCE_SUM);

Parent topic: Local Operations