Communication Engine

HCCL uses diverse communication engines to implement communication operators across different scenarios. The comparison is summarized below.

**Table 1** Application scenarios of different communication engines
Communication Engine	Advantage	Restriction	Scenario	Supported Product
AICPU+TS	No compute core is occupied, and the communication efficiency is high. It applies to scenarios with a huge amount of data and high bandwidth.	The static communication overhead is large, which is not friendly to communication scenarios with a small amount of data.	High-bandwidth communication	Atlas 350 Accelerator Card Atlas A3 training product / Atlas A3 inference product
Host CPU+TS	No compute core is occupied.	The dispatch overhead is large and increases linearly with the number of tasks.	N/A	Atlas A2 training product / Atlas A2 inference product
AIV	Low latency.	Communication consumes Vector cores, requiring multiple cores to saturate the communication bandwidth. Furthermore, resource contention between communication and computation operators may impact mutual performance.	Low-latency communication	Atlas 350 Accelerator Card
CCU	This engine reduces memory bandwidth overhead and Vector core utilization. It delivers high bandwidth and low latency.	A limited number of communicators are supported due to on-chip resources.	High-bandwidth and low-latency communication	Atlas 350 Accelerator Card

Below describes the task execution processes of different communication engines.

The AICPU submits communication-related tasks to the TS, as shown in Figure 1.

The host submits an AICPU kernel to the task queue.
The AICPU kernel is scheduled by the TS and then executed by the AICPU.
The AICPU submits a communication task to the task queue.
The communication task submitted by the AICPU is scheduled by the TS to the executor for execution.

Figure 1 AICPU+TS scheduling

The host CPU submits communication-related tasks to the TS on the device, as shown in Figure 2.

The host submits various operations (including memory copy and synchronization operations) in the communication process to the task queue.
The TS schedules the tasks dispatched to the task queue to the corresponding executor for execution.

Figure 2 Host CPU+TS scheduling

The Vector Core handles the execution logic and operation steps of the communication operator, as shown in Figure 3.

The host submits an AIV kernel to the task queue.
The AIV kernel is scheduled by the scheduler and then sent to the Vector Core for execution.
The Vector Core uses different protocols to transfer data.

Figure 3 AIV communication

The communication operator is executed by the Collective Communication Unit (CCU), as shown in the figure.

The host dispatches the CCU command sequence (composed of CCU-compatible instructions) to the CCU command space and submits the corresponding CCU kernel task to the task queue.

The CCU kernel is scheduled by the scheduler and then sent to the CCU for execution.
The CCU executes the corresponding instruction stream and uses Unified Remote Memory Access (URMA) to complete data transfer.

Figure 4 CCU acceleration

Parent topic: Programming Models and Concepts