Intended Audience

This document describes how to use the C++ APIs of the unilateral communication library to implement data transmission between clusters and build a disaggregated architecture for LLM inference, thereby improving the throughput performance of the LLM inference system.

To better understand this document, you are expected to:

• Have basic understanding of LLM structure and concepts.
• Be familiar with the LLM inference process.
• Be proficient in C++ programming.
• Be experienced in LLM inference frameworks.