--virtual_type

Applicability

Description

Indicates whether AOE can run on virtual devices generated on Ascend virtual instances.

If the computing power of a processor is too large to be used by cloud users or small enterprises, Ascend virtual instances can be applied to allocate certain computing power to the users or small enterprises to suit their services.

A virtual device is a virtual acceleration resource allocated by a processor based on specified computing power.

This option cannot be used in SGAT tuning.

See Also

None

Arguments

Arguments:

• 0: AOE does not run on virtual devices generated on Ascend virtual instances.
• 1: AOE runs on all types of virtual devices.

Restrictions: If the argument is 1, the model tuned by AOE is used. If it contains the operators listed in 3, a single core is used by default, which will degrade model inference performance.

Default: 0

Suggestions and Benefits

None

Examples

--virtual_type=1

Restrictions

Currently, only the Atlas inference products supports this option.

1. Before using this option, ensure that the Ascend virtual instance environment has been set up in the operating environment.
2. If virtual_type is set to 1 for AOE tuning, the number of running NPU cores in use may be greater than the actual number of cores specified by aicore_num. It is the least common multiple of the value range of aicore_num.

   For example, if the value range of aicore_num is {1,2,4,8}, the number of NPU cores is 8 when virtual_type is set to 1 during tuning.

3. When virtual_type is set to 1 and the model tuned by the AOE tool contains the following operators, a single core is used by default and the inference performance deteriorates.
   • DynamicRNN
   • PadV2D
   • SquareSumV2
   • DynamicRNNV2
   • DynamicRNNV3
   • DynamicGRUV