HcommWriteReduceWithNotifyOnThread

Applicability

Product	Supported
Atlas 350 Accelerator Card	√
Atlas A3 training product/Atlas A3 inference product	x
Atlas A2 training product/Atlas A2 inference product	x
Atlas 200I/500 A2 inference product	x
Atlas inference product	x
Atlas training product	x

Function

Writes data to the specified memory on a channel. It performs a reduceOp operation on the memory data of length count × sizeof(dataType) pointed to by src and the memory data of the same length pointed to by dst, outputs the result to dst, and sends a synchronization signal to the node where dst resides. The API caller is the node where src resides. This API is asynchronous.

Prototype

int32_t HcommWriteReduceWithNotifyOnThread(ThreadHandle thread, ChannelHandle channel, void *dst, const void *src, uint64_t count, HcommDataType dataType, HcommReduceOp reduceOp, uint32_t remoteNotifyIdx)

Parameters

Parameter	Input/Output	Description
thread	Input	Communication thread handle, which is threads returned by the HcclThreadAcquire call. For details, see ThreadHandle.
channel	Input	Communication channel handle, which is channels returned by the HcclChannelAcquire call. For details, see ChannelHandle.
dst	Output	Destination memory address, which is obtained using HcclGetHcclBuffer or HcclChannelGetHcclBuffer.
src	Input	Source memory address, which is obtained using HcclGetHcclBuffer or HcclChannelGetHcclBuffer.
count	Input	Number of elements.
dataType	Input	Data type. For details, see HcommDataType. Atlas 350 Accelerator Card: The supported data types are int8, int16, int32, uint8, uint16, uint32, float16, float32, and bfp16.
reduceOp	Input	Reduction operation type. For details, see HcommReduceOp. Atlas 350 Accelerator Card: The supported reduction types are sum, max, and min.
remoteNotifyIdx	Input	Notify index at the remote end of the communication channel. The value range is [0, notifyNum in the channelDescs parameter passed to the HcclChannelAcquire API).

Returns

int32_t: 0 on success; else, failure.

Restrictions

This API must be used in pair with HcommChannelNotifyWaitOnThread.

For the Atlas 350 Accelerator Card, this API can be called only on the device in AICPU_TS mode.

Example

// Allocate communication threads.
CommEngine engine = CommEngine::COMM_ENGINE_AICPU_TS;
uint32_t threadNum = 1;
uint32_t notifyNumPerThread = 1;
ThreadHandle thread;
HcclThreadAcquire(engine, threadNum, notifyNumPerThread, &thread);

// Allocate communication channels.
uint32_t channelNum = 1;
HcclChannelDesc channelDesc;
HcclChannelDescInit(&channelDesc, channelNum);
HcclComm comm;
ChannelHandle channel;
HcclChannelAcquire(comm, engine, &channelDesc, channelNum, &channel);

// Obtain the local communication memory information.
void * localBuffer;
uint64_t localBufferSize;
HcclGetHcclBuffer(comm, &localBuffer, &localBufferSize);

// Obtain the remote communication memory information.
void * remoteBuffer;
uint64_t remoteBufferSize;
HcclChannelGetHcclBuffer(comm, channel, &remoteBuffer, &remoteBufferSize);

// Copy parameters and launch the kernel.
// Orchestrate algorithms on the device.
uint64_t len = std::min(localBufferSize, remoteBufferSize);
uint64_t sizeOfFP32 = 4;
uint64_t count = len / sizeOfFP32;

// Write the content of the local memory to the remote memory and notify the remote end.
uint32_t rmtNotifyIdx = 0;
HcommWriteReduceWithNotifyOnThread(thread, channel, remoteBuffer, localBuffer, count, HCOMM_DATA_TYPE_FP32, HCOMM_REDUCE_SUM, rmtNotifyIdx);

// Data plane operations
// ...

// Wait for the remote end to notify the local end.
uint32_t lclNotifyIdx = 0;
uint32_t notifyTimeout = 0;
HcommChannelNotifyWaitOnThread(thread, channel, lclNotifyIdx, notifyTimeout);

Parent topic: Communication Operations