Introduction to the Unilateral Communication Library

The unilateral communication library provides simple, reliable, and efficient point-to-point data transfer for cluster scenarios through simple APIs, bridging multiple AI applications and transmission links. It can be used in various scenarios such as LLM PD disaggregation, RL post-training parameter switching, and model parameter caching.

Key Advantages

One-sided zero-copy communication: The unilateral communication library provides simple and reliable one-sided communication APIs. Once local memory data is ready, direct data transmission to remote memory can be completed through one-sided operations without any action required from the remote node. This provides core technical support for building scheduling mechanisms that overlap communication and computation. Meanwhile, the zero-copy capability enables direct data transmission between user memory spaces, avoiding redundant data movement, reducing memory bandwidth usage, and lowering memory capacity consumption.
Hardware difference shielding and multi-link high-speed interconnection across devices: The unilateral communication library masks underlying hardware differences across Ascend series chips, so users do not need to adapt code for different chip architectures. At the communication link layer, it natively supports high-speed interconnection protocols such as RDMA and HCCS, with communication bandwidth up to 119 GB/s. It enables seamless high-speed interconnection across heterogeneous devices, meeting low-latency and high-throughput requirements.
Minimalist API design and deep adaptation to open-source frameworks: Featuring a minimalist API design with only about 10 core calls, the unilateral communication library lowers the integration barrier for developers and provides C++ and Python language bindings. It has been deeply integrated with open-source frameworks such as Mooncake and DeepLink. Mainstream inference engines including vLLM and SGLang can directly invoke its APIs to implement efficient cross-device KV cache transmission, reducing memory access latency by 20% during LLM inference and improving inference throughput.

Software Architecture

Figure 1 shows the software architecture of the unilateral communication library.

Figure 1 Software architecture of the unilateral communication library

Core components:

HIXL Engine: As the core transmission engine, it provides basic transmission APIs and supports multiple memory transmission types such as D2D, D2H, and H2D. It is compatible with various transmission protocols including HCCS and RDMA, enabling high-speed and reliable data transmission. It natively supports multiple data link types and is suitable for complex scenarios in homogeneous and heterogeneous clusters. It can efficiently adapt to link changes and schedule resources under dynamic cluster scaling, forming a reliable communication foundation for overall cluster operation.
LLM-DataDist: Built on the HIXL Engine, it provides a set of data transmission APIs with KV cache semantics, enabling fast and flexible integration with inference engines such as vLLM and SGLang.

Parent topic: Introduction