Mesh

Description

Mesh is a basic algorithm in the full-mesh topology. It is a full mesh between NPUs. Any two NPUs can directly send and receive data.

The following figure shows the process of implementing the AllReduce operator using the Mesh algorithm. Each NPU concurrently reads data from or writes data to the remote end through multiple HCCS links, so that bidirectional bandwidth of the duplex interconnection link is used.

The time complexity of the Mesh algorithm is O(1).

Required Time Calculation

**Table 1** Time required for each operation in the Mesh algorithm
Operation	Time Required
Scatter	$\text{[math]}$
Gather	$\text{[math]}$
Broadcast	The implementation uses Scatter+AllGather, and the time required is as follows. $\text{[math]}$
Reduce	The implementation uses ReduceScatter+Gather, and the time required is as follows. $\text{[math]}$
ReduceScatter	$\text{[math]}$
AllGather	$\text{[math]}$
AllReduce	The implementation uses ReduceScatter+AllGather, and the time required is as follows. $\text{[math]}$

Parent topic: Collective Communication Algorithms