Intended Audience

This document describes how to use the Python APIs of the LLM-DataDist component in the Unilateral communication library to implement point-to-point data transmission and build a disaggregated architecture for LLM inference, thereby improving the throughput performance of the LLM inference system.

To better understand this document, you are expected to:

• Have basic understanding of LLM structure and concepts.
• Be familiar with the LLM inference process.
• Be proficient in Python programming.
• Be experienced in LLM inference frameworks such as vLLM.