--virtual_type
Description
Supports an offline model to run on a virtual device generated by the Ascend virtual instance feature.
If the computing power of a processor is too much for cloud users or small enterprises, the Ascend virtual instance feature can be applied to allocate a proper amount of computing power as needed by the users or small enterprises to suit their services.
A virtual device is a virtual acceleration resource allocated from a processor based on the allocation specification.
See Also
If --buffer_optimize is set to l1_optimize, this option cannot be used together with --virtual_type. Otherwise, an error is reported, indicating that L1 fusion is not performed in virtualization scenarios. This prevents scheduling exceptions caused by large operators.
Argument
- 0 (default): The offline model does not run on the virtual device generated by the Ascend virtual instance feature.
- 1: The offline model runs on virtual devices with different computing power.
Suggestions and Benefits
None
Example
--virtual_type=1
Applicability
Restrictions
- Before using this option, ensure that the Ascend virtual instance environment has been set up in the operating environment.
- For the MindSpore framework:
- The ReduceMean operator does not support the --virtual_type option.
- The ReverseV2 operator supports the --virtual_type option only in the
Atlas Training Series Product .
- If model conversion is performed with --virtual_type set to 1, the number of NPU blocks (blockDim) of the generated offline model may be greater than the actual number of aicore_num cores. The value is the least common multiple supported by aicore_num.
For example, if the value range of aicore_num is {1, 2, 4, 8}, the number of NPU blocks may be 8.
- If --virtual_type is set to 1 and the ATC generated model contains the following operators, a single core is used by default. In this case, the inference performance of the generated model deteriorates.
- DynamicRNN
- PadV2D
- SquareSumV2
- DynamicRNNV2
- DynamicRNNV3
- DynamicGRUV