Feature Description

The HDK-based virtual instance feature partitions the NPUs configured on a physical machine or virtual machine into multiple vNPUs (virtual NPUs) through resource virtualization, and mounts them into containers for use. The virtualization management enables the allocation and reclamation of resources in various specifications, accommodating repeated resource request and release operations from multiple users.

The Ascend HDK-based virtual instance function allows multiple users to share a single server on demand, lowering the entry barrier and cost of accessing NPU computing power. By enabling resource isolation through containers, this approach ensures a stable and secure runtime environment. The unified resource allocation and reclamation process also facilitates multi-tenant management.

Principles

Ascend NPU hardware resources mainly include AICore (used for AI model computation), AICPU, memory, etc. The main principle of the HDK-based virtual instance function is to partition the hardware resources into vNPUs according to user‑specified resource requirements, with each vNPU corresponding to a set of AICores, AICPUs, and memory resources. For example, when a user needs only 4 AICores of computing power, the system creates a vNPU, through which it obtains 4 AICores from the NPU and provides them to the container. The HDK-based virtual instance solution is shown in Figure 1.

Figure 1 Virtual instance solution based on HDK

Products Support Notes

Table 1 Product support notes

Product Series	Supported Scenarios	Virtualization Mode	Supported
Atlas Inference Series Products Atlas 300I Pro Inference Card Atlas 300V Video Analysis Card Atlas 300V Pro Video Analysis Card Atlas 300I Duo Inference Card Atlas 200I SoC A1 Core Board	Partition vNPUs on the physical machine, and mount vNPUs to the container.	Static Virtualization	Yes
Atlas Inference Series Products Atlas 300I Pro Inference Card Atlas 300V Video Analysis Card Atlas 300V Pro Video Analysis Card Atlas 200I SoC A1 Core Board	Partition vNPUs on the physical machine, and mount vNPUs to the container.	Dynamic Virtualization	Yes
Atlas Inference Series Products Atlas 300I Pro Inference Card Atlas 300V Video Analysis Card Atlas 300V Pro Video Analysis Card Atlas 300I Duo Inference Card	Partition vNPUs on the physical machine, and mount vNPUs to the virtual machine.	Static Virtualization	Yes
	Partition vNPUs on the physical machine, mount vNPUs to the virtual machine, and then mount vNPU to containers within the virtual machine.	Static Virtualization	Yes
	Pass through the NPU from the physical machine to the virtual machine, partition vNPUs within the virtual machine, and then mount the vNPUs to containers inside the virtual machine.	Static Virtualization	Yes
Atlas Inference Series Products Atlas 300I Pro Inference Card Atlas 300V Video Analysis Card Atlas 300V Pro Video Analysis Card	Pass through the NPU from the physical machine to the virtual machine, partition vNPUs within the virtual machine, and then mount the vNPUs to containers inside the virtual machine.	Dynamic Virtualization	Yes
Atlas 800 Training Server	Partition vNPUs on the physical machine, and mount vNPUs to the virtual machine.	Static Virtualization	Yes
Atlas Training Series Products Atlas 300T Training Card (Model 9000) Atlas 300T Pro Training Card (Model 9000) Atlas 800 Training Server (Model 9000) Atlas 800 Training Server (Model 9010) Atlas 900 PoD (Model 9000) Atlas 900T PoD Lite	Partition vNPUs on the physical machine, and mount vNPUs to the container.	Static Virtualization	Yes
Atlas A2 Training Series Products Atlas 800T A2 Training Server (24 AICores)	Partition vNPUs on the physical machine, and mount vNPUs to the container.	Static Virtualization Dynamic Virtualization	Yes
Atlas A3 Training Series Products Atlas 800T A3 SuperPoD Server	Partition vNPUs on the physical machine, and mount vNPUs to the container.	Static Virtualization Dynamic Virtualization	Yes
Atlas A2 Inference Series Products Atlas 800I A2 Inference Server	Partition vNPUs on the physical machine, and mount vNPUs to the container.	Static Virtualization Dynamic Virtualization	Yes
Atlas A3 Inference Series Products Atlas 800I A3 SuperPoD Server	Partition vNPUs on the physical machine, and mount vNPUs to the container.	Static Virtualization Dynamic Virtualization	Yes
Atlas 200/300/500 Inference Products	-	-	No
Atlas 200I/500 A2 Inference Products	-	-	No

Usage Instructions

Static virtualization and dynamic virtualization are implemented based on HDK. The NPU is partitioned into vNPUs through HDK interfaces and then mounted to containers for use.
If you are using dynamic virtualization, directly refer to the "Dynamic vNPU Scheduling (Inference)" section. You do not need to use the npu-smi command to create vNPUs in advance.
If you are using static virtualization, you need to first refer to "Creating vNPU", and then perform the operation of mounting them to a container.
For detailed descriptions of npu-smi commands, see the "Ascend Virtual Instance (AVI) Commands" section in the Atlas A3 Center Inference and Training Hardware 26.0.RC1 npu-smi Command Reference.

Usage Constraints

After a physical NPU is virtualized into vNPUs, it is no longer supported to mount the physical NPU to a container, nor is it supported to pass through the physical NPU to a virtual machine.
A vNPU can only be used by one task container. It is not supported for multiple task containers to use the same vNPU.
The operating modes of the two chips on the Atlas 300I Duo inference card must be consistent. That is, both chips must use the virtualization instance function, or the entire card must use this function. Plan according to your service requirements.
The virtual instance template is used to partition resources across all NPUs on an entire server. Mixing cards of different specifications is not supported. For example, the Atlas 300V Pro video analysis card supports 24 GB and 48 GB memory specifications, virtualization does not support mixing these two memory specifications on the same server. Similarly, mixing Atlas training series products with 30 AICores and those with 32 AICores is not supported.
For Atlas training series servers, the virtual instance function is supported only when the NPU operates in AMP mode, not in SMP mode. The steps for querying and setting the NPU working mode are as follows (ensure the server operating system is powered off).
1. Log in to the iBMC command line.
2. Run the ipmcget -d npuworkmode command to query the NPU working mode. If it is AMP mode, no switching is required.
3. Run the ipmcset -d npuworkmode -v 0 command to set the NPU working mode to AMP mode.

For detailed instructions on querying and setting the NPU working mode, see the "Command Line Introduction > Server Commands > Querying and Setting the NPU Working Mode (npuworkmode)" in the Atlas 800 Training Server iBMC User Guide (model 9000).