Intended Audience

This document describes how to use the C++ APIs of LLM-DataDist to implement data transmission between clusters and build a disaggregated architecture for LLM inference, thereby improving the throughput performance of the LLM inference system.

To better understand this document, you are expected to have the following:

• Basic understanding of LLM structure and concepts
• Familiarity with the LLM inference process
• Proficiency in C++ programming
• Experience using LLM inference frameworks