--op_precision_mode
Applicability
Product |
Supported |
|---|---|
√ |
|
√ |
|
√ |
|
√ |
|
√ |
Description
Sets the precision mode of an operator. You can use this option to set different precision modes for different operators.
See Also
This option is mutually exclusive with --op_select_implmode and --optypelist_for_implmode. If they are used together, only the mode specified by --op_precision_mode takes effect.
Arguments
Argument: directory (including the file name) of the .ini configuration file. The following precision mode is supported:
- high_precision
- high_performance
- enable_float_32_execution: The FP32 data type is used for internal processing of operators. In this scenario, the FP32 data type is not converted to the HF32 data type. If you are using the HF32 data type for computation and find that the accuracy drop exceeds your expectation, you can enable this configuration to specify the use of FP32 for internal computation of certain operators in order to maintain accuracy. This configuration is supported only in the
Atlas A2 training products /Atlas A2 inference products scenario. - enable_float_32_execution: The FP32 data type is used for internal processing of operators. In this scenario, the FP32 data type is not converted to the HF32 data type. If you are using the HF32 data type for computation and find that the accuracy drop exceeds your expectation, you can enable this configuration to specify the use of FP32 for internal computation of certain operators in order to maintain accuracy. This configuration is supported only in the
Atlas A3 training products /Atlas A3 inference products scenario. - enable_hi_float_32_execution: The HF32 data type is used for internal processing of operators. After this configuration is enabled, the FP32 data type is converted to the HF32 data type. This configuration can reduce the space occupied by data and improve performance. This is not supported in the current version.
- enable_hi_float_32_execution: The HF32 data type is used for internal processing of operators. After this configuration is enabled, the FP32 data type is converted to the HF32 data type. This configuration can reduce the space occupied by data and improve performance. This is not supported in the current version.
- support_out_of_bound_index: indicates that the out-of-bounds verification is performed on the indices of the gather, scatter, and segment operators. The verification deteriorates the operator execution performance.
You can view the precision or performance mode supported by an operator in the opp/built-in/op_impl/ai_core/tbe/impl_mode/all_ops_impl_mode.ini file in the file storage path with the CANN software installed.
Format: The directory (including the file name) can contain letters, digits, underscores (_), hyphens (-), periods (.), and Chinese characters.
Restrictions:
- Currently, the operator precision can be set only by using the .ini configuration file. The configuration file content is formatted as operator type=precision mode (key-value pairs). The precision mode of an operator is set in each row.
- The operator type must be OpType of the Ascend IR–defined operator. For details about how to view the operator type, see How Do I Determine the Mapping Between Operators in the Original Network Model and Operators Supported by Ascend AI Processor?.
Suggestions and Benefits
- You are advised not to set this option. It is used if you need to adjust the precision of a specific operator using the .ini configuration file in the case that you fail to obtain optimal network performance or accuracy in high-performance or high-precision mode.
- You are advised to load the .ini configuration file saved after the --op_select_implmode option is configured. For details, see Suggestions and Benefits.
Examples
Build the operator precision mode configuration file op_precision.ini and set the operator precision mode in the file. Set the precision mode of an operator each row. The following is an example:
optype1=high_precision optype2=high_performance
Upload the configured op_precision.ini file to any directory (for example, ${HOME}/conf) on the server where AOE is located.
--op_precision_mode=${HOME}/conf/op_precision.ini
Dependencies and Restrictions
None