Pipeline

Description

To mitigate network traffic conflicts, AI computing clusters adopt hierarchical network architectures. Specifically, intra-server interconnection is established via direct cabling, whereas inter-server communication between logical devices with identical IDs is facilitated through switches. To adapt to this network topology, the collective communication uses a hierarchical algorithm policy, which decomposes global communication operations into multiple levels of local operations, and uses a phased, layer-by-layer progressive approach to improve communication efficiency.

Taking the AllGather operator as an example, an AllGather operation is first executed across servers among logical devices with identical IDs, followed by another AllGather operation within each server, thereby completing the data collection process for the entire cluster. However, this approach causes a certain degree of link bandwidth waste: when transferring data between servers, the intra-server links remain idle, failing to fully utilize the bandwidth resources.

To resolve this issue, HCCL adopts a fine-grained hierarchical pipeline algorithm. By exploiting the data dependencies inherent in the communication algorithm and combining them with pipeline parallelism, it addresses the problem of insufficient bandwidth utilization.

Take AllGather as an example. The Ring algorithm is used for inter-server communication, and the FullMesh algorithm is used for intra-server communication, as shown in the following figure.

As shown in the preceding figure, the green data chunk is sent from rank 5 to rank 1 (only the behavior of some ranks is described here, and other ranks are symmetrically processed). In the next step, rank 1 continues to send the green data chunk to rank 3 (standard step of the Ring algorithm), and rank 1 can also send the green data chunk to rank 0 within the same server. The execution of the Ring algorithm continues. In each step, data is transmitted between servers, and the data chunk received in the previous step is transmitted to other ranks within the server. After the last step of the Ring algorithm is complete, only one data chunk transmission needs to be performed within the server to complete all algorithm steps. (The initial data chunk of the rank can be transmitted within the server in the first step of the Ring algorithm.)

The following figure shows the orchestration of all transmission tasks from the perspective of rank 0. The local copy operation is performed only when the input memory and output memory are different. It is used to move data chunks from the input memory to the output memory. If the input memory and output memory are the same, this operation is not required.

Required Time Calculation

p indicates the total number of devices for completing collective communication, indicates the number of servers, s indicates the total data size of collective communication operations, indicates the time required for transmitting each byte of data over the inter-server link, indicates the time required for transmitting each byte of data over the intra-server link, indicates the fixed time required for transmitting data over the inter-server link, and indicates the fixed time required for transmitting data over the intra-server link.