Point-to-point communication involves directly transmitting data between two NPUs in a multi-NPU system. This method is typically used to transmit and receive activation values in pipeline parallel scenarios.

HCCL provides P2P communication operators at different granularities, including the single-rank TX and RX operators (Send/Receive) as well as the inter-rank batch TX and RX operator (BatchSendRecv). HCCL also provides corresponding APIs for you to call.

Send/Receive (Single TX/RX)

Send: sends data of a rank to another rank.
Receive: receives data sent from another rank.

HCCL provides the HcclSend and HcclRecv APIs for the single RX and TX scenarios. They must be dispatched in strict order and used in pairs. Data can be sent and received only after synchronization is complete between the TX and RX ends. Subsequent operator tasks can be executed only after data is sent and received.

Click to enlarge

A simple code snippet is as follows:

      
           if(rankId == 0){
    uint32_t destRank = 1;
    uint32_t srcRank = 1;
    HcclSend(sendBuf, count, dataType, destRank, hcclComm, stream);
    HcclRecv(recvBuf, count, dataType, srcRank, hcclComm, stream);
}
if(rankId == 1){
    uint32_t srcRank = 0;
    uint32_t destRank = 0;
    HcclRecv(recvBuf, count, dataType, srcRank, hcclComm, stream);
    HcclSend(sendBuf, count, dataType, destRank, hcclComm, stream);
}

BatchSendRecv (Batch RX and TX)

HCCL provides the HcclBatchSendRecv API for data RX and TX among multiple ranks in a communicator. This API has the following features:

The API rearranges the sequence of sending and receiving data in batches. Therefore, the sequence of sending and receiving data in batches in a single API call is not strictly required. Ensure that the number of data sending operations in a single API call is equal to the number of data receiving operations.
The sending and receiving processes are independently scheduled and executed, and they do not block each other, thereby implementing duplex link concurrency.

Note: In a single API call, only one memory block can be transferred in a unidirectional data flow between two ranks. This prevents the RX and TX addresses of multiple memory blocks from being confused during data sending and receiving.

A simple code snippet is as follows:

      
       
         
         
           HcclSendRecvItem sendRecvInfo[itemNum];
HcclSendRecvType currType;
for (size_t i = 0; i < op_type.size(); ++i) {
    if (op_type[i] == "isend") {
        currType = HcclSendRecvType::HCCL_SEND;
    } else if (op_type[i] == "irecv") {
        currType = HcclSendRecvType::HCCL_RECV;
    } 
    sendRecvInfo[i] = HcclSendRecvItem{currType,
                                       tensor_ptr_list[i],
                                       count_list[i],
                                       type_list[i],
                                       remote_rank_list[i]
                                       };
}
HcclBatchSendRecv(sendRecvInfo, itemNum, hcclComm, stream);

          

        

      
     

Point-to-Point Communication

Send/Receive (Single TX/RX)

BatchSendRecv (Batch RX and TX)