Before You Start
MindCluster supports deploying SGLang inference jobs through OME (Open Model Engine) for scheduling and fault instance rescheduling.
This chapter describes the relevant feature principles and corresponding configuration examples. You can refer to the configuration examples to deploy OME-based SGLang inference jobs.
Before deploying the SGLang inference service, ensure that the relevant components have been installed. If they are not installed, refer to the Installation and Deployment chapter for instructions.
- Volcano
- Ascend Device Plugin
- Ascend Docker Runtime
- ClusterD
- NodeD (Optional)
- Atlas 800I A2 inference server
- Atlas 800I A3 SuperPoD server
MindCluster supports containerized deployment and fault rescheduling of SGLang inference service through the following methods. This section only introduces the command-line usage and the one-click script deployment method.
- Using via Command Line: Deploy jobs through configured YAML files.
- Using via One-Click Script Deployment: Deploy jobs through automated script reference designs.
- Using after integration: Integrate cluster scheduling components into an existing third-party AI platform or an AI platform developed based on these components.