Introduction

Large language model DataDist (LLM-DataDist) provides C++ and Python APIs to manage KV data in a cluster, enabling separate deployment of full graphs and incremental graphs.

**Table 1** Use cases
Manual	Description
LLM-DataDist API Reference (C++)	Describes LLM-DataDist APIs for C++, including link management and KV cache management. This scenario supports one-way link establishment (that is, the client initiates link establishment to the server). Data transmission is restricted to pulling KV cache from Decode to Prompt and pushing KV cache from Prompt to Decode. Only D2D transmission is supported in this scenario.
LLM-DataDist API Reference (Python)	Describes LLM-DataDist APIs for Python, including link management and KV cache management in KvCacheManager mode. This scenario supports one-way link establishment. Data transmission is restricted to pulling KV cache from Decode to Prompt and pushing KV cache from Prompt to Decode. Only D2D transmission is supported in this scenario.