Inference

Table 1 Inference job scenario

Installation Scenario

Component

Job Type

Operations to Be Performed in a Deployment Job

Description

Full deployment scenario or cluster scheduling scenario

Ascend Docker Runtime

Ascend Device Plugin

Volcano

(Optional) NPU-Exporter

  • Job (Kubernetes' resource object)
  • Deployment (Kubernetes' resource object)

Familiarize yourself with the basic process described in NPU Inference Job. Then, try the subsequent sections, for example, delivering NPU inference jobs using the CLI or by programming. Finally, integrate advanced features.

The component list contains only the minimum set of components that need to be installed to support the inference function in the installation and deployment scenario.

Device management scenario

Ascend Docker Runtime

Ascend Device Plugin (The startup parameter volcanoType is set to false.)

(Optional) NPU-Exporter