Reading Guide

If you are not familiar with LLM-DataDist or are reading this document for the first time, you are advised to read Table 1 first to quickly learn the manual content and access the sections as needed.

Table 1 Reading guide

Scenario

Section

Content

Understanding LLM-DataDist

Introduction

This section describes the basic concepts, functions, and advantages of LLM-DataDist, and introduces its application scenarios.

Building a disaggregated LLM inference framework based on LLM-DataDist

Quick Start

This section describes the overall development process and environment preparation for building a disaggregated LLM inference framework using LLM-DataDist, and provides complete executable reference samples.

Learning about LLM-DataDist API capabilities

Function Description

This section describes the functions of LLM-DataDist APIs and how to implement them.

Acquiring extended knowledge

Appendix

This section describes extended knowledge mentioned in this document, including basic concepts and LLM inference process.

Troubleshooting

Error Codes

This section describes API error codes, along with cause analysis and solutions.