Intended Audience

This document describes how to use the Python APIs of LLM-DataDist to implement point-to-point data transmission and build a disaggregated architecture for LLM inference, thereby improving the throughput performance of the LLM inference system.

To better understand this document, you are expected to have the following:

• Basic understanding of LLM structure and concepts
• Familiarity with the LLM inference process
• Proficiency in Python programming
• Experience using LLM inference frameworks such as vLLM