HcommWriteWithNotifyOnThread

Applicability

Product	Supported
Atlas 350 Accelerator Card	√
Atlas A3 training product/Atlas A3 inference product	x
Atlas A2 training product/Atlas A2 inference product	x
Atlas 200I/500 A2 inference product	x
Atlas inference product	x
Atlas training product	x

Function

Writes data to the specified memory on a channel. It writes memory data of length len from src into the memory of the same length pointed to by dst, and sends a synchronization signal to the node where dst resides. The API caller is the node where src resides. This API is asynchronous.

Prototype

int32_t HcommWriteWithNotifyOnThread(ThreadHandle thread, ChannelHandle channel, void *dst, const void *src, uint64_t len, uint32_t remoteNotifyIdx)

Parameters

Parameter	Input/Output	Description
thread	Input	Communication thread handle, which is threads returned by the HcclThreadAcquire call. For details, see ThreadHandle.
channel	Input	Communication channel handle, which is channels returned by the HcclChannelAcquire call. For details, see ChannelHandle.
dst	Output	Destination memory address, which is obtained using HcclGetHcclBuffer or HcclChannelGetHcclBuffer.
src	Input	Source memory address, which is obtained using HcclGetHcclBuffer or HcclChannelGetHcclBuffer.
len	Input	Data length (in bytes).
remoteNotifyIdx	Input	Notify index at the remote end of the communication channel. The value range is [0, notifyNum in the channelDescs parameter passed to the HcclChannelAcquire API).

Returns

int32_t: 0 on success; else, failure.

Restrictions

This API must be used in pair with HcommChannelNotifyWaitOnThread.

For the Atlas 350 Accelerator Card, this API can be called only on the device in AICPU_TS mode.

Example

// Allocate communication threads.
CommEngine engine = CommEngine::COMM_ENGINE_AICPU_TS;
uint32_t threadNum = 1;
uint32_t notifyNumPerThread = 1;
ThreadHandle thread;
HcclThreadAcquire(engine, threadNum, notifyNumPerThread, &thread);

// Allocate communication channels.
uint32_t channelNum = 1;
HcclChannelDesc channelDesc;
HcclChannelDescInit(&channelDesc, channelNum);
HcclComm comm;
ChannelHandle channel;
HcclChannelAcquire(comm, engine, &channelDesc, channelNum, &channel);

// Obtain the local communication memory information.
void * localBuffer;
uint64_t localBufferSize;
HcclGetHcclBuffer(comm, &localBuffer, &localBufferSize);

// Obtain the remote communication memory information.
void * remoteBuffer;
uint64_t remoteBufferSize;
HcclChannelGetHcclBuffer(comm, channel, &remoteBuffer, &remoteBufferSize);
uint64_t len = std::min(localBufferSize, remoteBufferSize);

// For the Atlas 350 Accelerator Card, the following APIs need to be called on the device:

// Write the content of the local memory to the remote memory and notify the remote end.
uint32_t rmtNotifyIdx = 0;
HcommWriteWithNotifyOnThread(thread, channel, remoteBuffer, localBuffer, len, rmtNotifyIdx);

// Data plane operations
// ...

// Wait for the remote end to notify the local end.
uint32_t lclNotifyIdx = 0;
uint32_t notifyTimeout = 0;
HcommChannelNotifyWaitOnThread(thread, channel, lclNotifyIdx, notifyTimeout);

Parent topic: Communication Operations