Sample Reference

This sample demonstrates how to use LLM-DataDist APIs to manage the KV cache in a disaggregated deployment scenario, providing references and adaptation solutions for integrating LLM-DataDist into different frameworks.

Try now: Click Gitee, download the sample package of the matching version based on the tag name, and obtain the sample from the cplusplus/level1_single_api/11_llm_data_dist directory.