Communication Model

Figure 1 HCCL communication model

The preceding figure shows the HCCL communication model. The involved concepts are all software concepts, which are described in the following:

  • Communication memory: A memory block on a communication member can be registered with a communicator, indicating that the memory block can be accessed by other communication members in the communicator.
  • Endpoint: a port (such as NetDevice of a NIC) used for communication with other communication objects.
    • An endpoint contains attributes such as the address and protocol, and may encompass multiple physical ports (for example, in the bonding scenario).
    • Each communication object can contain multiple endpoints.
  • Channel: a communication channel established between the specified endpoint of a communication object and the specified endpoint of another communication object.
    • Multiple channels can be established between a pair of endpoints.
    • When a channel is created, the local and remote ends need to call the channel creation API synchronously.
    • When a channel is created, the memory information registered by the local end is exchanged with that registered by the remote end. The control plane also provides an API for querying the remote memory information (address and size) based on the channel.
Communication models are classified into the network semantic communication model and memory semantic communication model.
  • Network semantic communication model: The communication operator developer accesses the memory of the remote communication object based on channels or synchronize with the remote communication object.
  • Memory semantic communication model: The communication memory of a remote object can be mapped to the local memory address space. The communication operator developer can directly use the local memory copy operation to access the remote communication memory.

Network Semantic Communication Model

In the network semantic communication model, channels are used to read and write the memory of remote communication objects or synchronize with remote communication objects. For details, see the API description in Communication Operations.

Figure 2 Network semantic communication model

Channels are key objects of the network semantic communication model. You can create multiple channels between a pair of endpoints. The following describes the internal elements of the channel model.

Figure 3 shows the channel model in the RoCE scenario.

  • A channel is the entry for communication between the local and remote objects. The channel of the local communication object is associated with the channel of the remote communication object in a one-to-one manner.
  • A channel is associated with one or more queue pairs (QPs) which are mapped with QPs associated with the remote channel. When a channel is established, links are established between the corresponding QPs.
  • A channel contains multiple Notify instances, which are used for synchronization between communication objects. Notify is an abstract concept for synchronization operations and may be implemented by different entities in different communication engines.
    • When creating a channel, you can specify the number of Notify instances contained in the channel.
    • The local end can use the channel to send a synchronization signal to a Notify instance (specified by a sequence number) in the channel corresponding to the remote object. For details, see Data Plane APIs.
    • The local end can wait for a synchronization signal from the remote communication object based on a Notify instance of the channel. Subsequent operations can be performed only after the synchronization signal is received.
Figure 3 Channel model in the RoCE scenario

Memory Semantic Communication Model

In the memory semantic communication model, the communication memory of a remote object can be mapped to the address space of a local process. The communication operator developer can use local operation APIs to implement data movement or synchronization between nodes. For details, see Local Operations.

The following figure shows the memory semantic communication model of HCCL.

  • Endpoint: a network logical port used for communication with other communication objects.
    • In the memory semantic communication model, a communication object has only one endpoint.
    • An endpoint contains attributes such as the address and protocol, and may encompass multiple physical ports (for example, in the bonding scenario).
  • Channel: a communication channel established between the local and remote communication objects. In the memory semantic scenario, the establishment of a channel indicates that the memory mapping function between two communication objects is enabled and the channel is not used for communication between communication objects.
    • Only one channel can be created between a pair of communication objects.
    • When a channel is created, the memory information registered by the local end is exchanged and mapped with that registered by the remote end.
    • The control plane provides the function of querying the address and size of the remote memory mapped to the local end based on the channel.
Figure 4 Memory semantic communication model