Creating Resources

Communication Resource Computing

Communication operators depend on underlying hardware communication resources during execution. Therefore, before orchestrating and dispatching operator tasks, allocate required communication resources. The communication resources required by the AIV communication operators are classified into the following types:

Vector Core: expresses the parallelism relationship within a single rank. During algorithm orchestration, different ranks or data blocks can be partitioned and processed in parallel across multiple compute cores. Multiple AI Cores share the same code logic, where the only distinction between the running instances on each compute core is their unique core IDs (indexes). Each core determines its own algorithmic branching workflow based on this specific index.
Notify: used to implement the synchronization mechanism, including intra-rank synchronization between different Vector Cores and inter-rank synchronization. Since the AI Core cannot invoke hardware features to achieve inter-rank synchronization, this functionality must be simulated via software algorithms, a process referred to as software synchronization. The Notify used in software synchronization is essentially a block of on-chip memory.
Channel: used for cross-rank data transfer.

Generally, the type and number of required communication resources vary with the communication algorithm. Using the single-server 4-device topology with the AIV communication engine as an example, this section describes the compute logic of the communication resources required by the mesh algorithm.

The following figure shows the hardware topology of the mesh algorithm. Each rank directly communicates with all other ranks.

The number of communication resources required by the mesh communication algorithm in the preceding networking is as follows:

Vector Core: Each rank needs to communicate with all other ranks. The communication tasks on each rank (including the local copy of the rank) are distributed across different cores for asynchronous execution. Therefore, each rank consumes a total of four Vector Core resources, which are automatically allocated by the system during operator launch without requiring pre-request.
Notify: Each rank synchronizes with all other ranks using paired pre-sync/post-sync operations. Therefore, each rank requires a total of six Notify resources. Each soft sync Notify resource is allocated with 32 bytes of on-chip memory, meaning that a reserved space ranging from approximately 1 KB to 1 MB is typically sufficient.
Channel: Each rank communicates with all other ranks. Therefore, each rank requires three communication channels.

Sample Code

The following uses the AllGather communication operator as an example to describe the code snippet for creating resources on the host.

Allocate the context memory to store resource information.

uint64_t size = size of the context memory to be allocated;
void *ctx = nullptr;
char *tag = tag used to save resources;
CommEngine engine = COMM_ENGINE_CPU_TS; // Use the memory on the host to cache resources.
HcclResult ret = HcclEngineCtxGet(comm, tag, engine, &ctx, &size);
if (ret != HCCL_SUCCESS) {
   // Resource for the tag has not been created yet.
   HcclEngineCtxCreate(comm, tag, engine, size, ctx); 
}else {
   // Resource has been created before. Use ctx directly.
}

Allocate Notify resources.

char tag[] = "allgather";
CommEngine engine = CommEngine::COMM_ENGINE_AIV;                   // AIV communication engine
void* aivTagBufPtr = nullptr;                                      //Address of the soft sync tag buffer
HcclEngineCtxCreate(comm, tag, engine, AIV_TAG_BUFF_LEN, &aivTagBufPtr); // Create Notify resources.
aclrtMemset(aivTagBufPtr, AIV_TAG_BUFF_LEN, 0, AIV_TAG_BUFF_LEN); // Clear the tag buffer.

HcclMemHandle memHandle;
CommMem regMem{COMM_MEM_TYPE_DEVICE, aivTagBufPtr, AIV_TAG_BUFF_LEN};
HcclCommMemReg(comm, tag, &regMem, &memHandle); // Register the soft-sync memory to the communicator.

HcclChannelDesc channelDesc;
HcclChannelDescInit(&channelDesc, 1);
channelDesc.remoteRank = destRank;
channelDesc.channelProtocol = CommProtocol::COMM_PROTOCOL_UB_MEM;
channelDesc.memHandles = &memHandle;           // Specify and exchange soft-sync memory during channel establishment. 
channelDesc.memHandleNum = 1;
ChannelHandle channelHandle;
HcclChannelAcquire(comm, engine, &channelDesc, 1, &channelHandle);

Allocate communication memory.

// Allocate and obtain the local communication memory.
void *localAddr;
uint64_t localSize;
HcclGetHcclBuffer(comm, &localAddr, &localSize);
// Obtain the remote communication memory for subsequent reads and writes.
void *remoteAddr;
uint64_t remoteSize;
HcclChannelGetHcclBuffer(comm, channelHandle, &remoteAddr, &remoteSize);
//Obtain the memory of the remote soft-sync tag buffer for subsequent synchronizations.
uint32_t memNum;
CommMem* remoteMems;
char** memTags;
HcclChannelGetRemoteMems(comm, channelHandle, &memNum, &remoteMems, &memTags);
void *remoteTagBufAddr = remoteMems[0].addr;

Parent topic: AIV Operator Development