--virtual_type

Applicable Products

All processors

Description

Sets whether to allow an offline model to run on a virtual device generated by the Ascend virtual instance feature.

For cloud users or small enterprises that do not need the full computing power of a processor, the Ascend virtual instance feature can allocate computing resources on demand to better suit their service requirements.

The Ascend virtual instance feature is used in the NPU resource virtualization scenario to improve the utilization of physical NPU resources through isolated vNPU instances. A virtual device is a virtual acceleration resource allocated from a processor based on the allocation specification.

Arguments

• 0 (default): The offline model does not run on the virtual device generated by the Ascend virtual instance feature.
• 1: The offline model runs on virtual devices with different computing power.

Suggestions and Benefits

None

Example

--virtual_type=1

Restrictions

• Before using this option, ensure that the Ascend virtual instance environment has been set up in the operating environment.
• For the MindSpore framework:
  • The ReduceMean operator does not support the --virtual_type option.
  • The ReverseV2 operator supports the --virtual_type option only in the Atlas inference products and Atlas training products.
• If model conversion is performed with --virtual_type set to 1, the number of logical AI Cores involved in the computation of the generated offline model may be greater than the actual number of cores specified by aicore_num. The value is the least common multiple of all valid configurable values for aicore_num.

  For example, if the value range of aicore_num is {1, 2, 4, 8}, the number of logical AI Cores involved in the computation is 8 for the offline model generated with --virtual_type set to 1.

• If --virtual_type is set to 1 and the ATC generated model contains the following operators, a single core is used by default. In this case, the inference performance of the generated model deteriorates.
  • DynamicRNN
  • PadV2D
  • SquareSumV2
  • DynamicRNNV2
  • DynamicRNNV3
  • DynamicGRUV

  This restriction does not apply to the Ascend 950PR/Ascend 950DT.