--virtual_type

Applicability

Description

Sets whether to allow an offline model to run on a virtual device generated by the Ascend virtual instance feature.

If the compute power of a processor is too much for cloud users or small enterprises, the Ascend virtual instance feature can be applied to allocate a proper amount of compute power as needed by the users or small enterprises to suit their services.

The Ascend virtual instance is used in the NPU resource virtualization scenario, which aims to improve the utilization of physical NPU resources through isolated vNPU instances. A virtual device is a virtual acceleration resource allocated from a processor based on the allocation specification.

See Also

None

Argument

• 0 (default): The offline model does not run on the virtual device generated by the Ascend virtual instance feature.
• 1: The offline model runs on virtual devices with different computing power.

Suggestions and Benefits

None

Example

--virtual_type=1

Restrictions

• Before using this option, ensure that the Ascend virtual instance environment has been set up in the operating environment.
• For the MindSpore framework:
  • The ReduceMean operator does not support the --virtual_type option.
  • The ReverseV2 operator supports the --virtual_type option only in the Atlas inference products and Atlas training products.
• If model conversion is performed with --virtual_type set to 1, the number of computing logical AI Cores of the generated offline model may be greater than the actual number of cores specified by aicore_num. The value is the least common multiple supported by aicore_num.

  For example, if the value range of aicore_num is {1, 2, 4, 8}, the number of NPU blocks is 8 for the offline model generated with --virtual_type set to 1.

• If --virtual_type is set to 1 and the ATC generated model contains the following operators, a single core is used by default. In this case, the inference performance of the generated model deteriorates.
  • DynamicRNN
  • PadV2D
  • SquareSumV2
  • DynamicRNNV2
  • DynamicRNNV3
  • DynamicGRUV