Before You Start
Prerequisites
Before using dynamic vNPU scheduling on the CLI, ensure that the following components have been installed. If not, refer to Installation and Deployment for further operations. Only Volcano can be used as the scheduler to allow for the dynamic vNPU scheduling feature.
- Volcano
- Ascend Device Plugin
- Ascend Docker Runtime
- ClusterD
- NodeD
Usage Modes
The dynamic vNPU scheduling feature can be used in either of the following modes:
- Use on the CLI: Install cluster scheduling components and enable dynamic vNPU scheduling through the CLI.
- Use after integration: Integrate the cluster scheduling components into an existing third-party AI platform or an AI platform developed based on the cluster scheduling components.
Instructions
- Resource monitoring can be used together with all features in inference scenarios.
- Multiple inference jobs can be run in a cluster at the same time. Each job can use different features, but jobs that support static vNPUs and jobs that support dynamic vNPUs cannot coexist.
- The dynamic vNPU scheduling feature can be used in conjunction with the computing power virtualization feature. For detailed descriptions and operations of dynamic virtualization, see Dynamic Virtualization.
- Dynamic vNPU scheduling supports only single-server jobs with a single replica or multiple replicas. Each replica works independently. Distributed jobs are not supported.
Supported Products
Atlas inference product
Usage Process
For details about how to use dynamic vNPU scheduling through CLI, see Figure 1.
For details about how to modify the parameters of related cluster scheduling components, see Dynamic Virtualization.
Parent topic: Dynamic vNPU Scheduling (Inference)