--op_select_implmode

Description

The function specified by this option will no longer evolve and the option will be deprecated in later versions. You are advised to use --op_precision_mode instead.

Selects an operator implementation mode. Certain operators built in the AI processor can be implemented in either high-precision or high-performance mode at model build time.

High-precision mode means that for float16 inputs, operator precision is further improved by means of Taylor expansion, Newton-Raphson iteration, and other techniques. High-performance mode refers to the implementation of optimal performance under float16 input conditions without compromising network precision.

Arguments

high_precision: High-precision mode.
This option sets the operator implementation mode by using the built-in configuration file, which is stored in ${INSTALL_DIR}/opp/op_impl/built-in/ai_core/tbe/impl_mode/high_precision.ini.

To ensure compatibility, this argument takes effect only for the operator list in the high_precision.ini file. This list can be used to control the effective scope of operators and ensure that the network models of earlier versions are not affected.
high_performance (default): high-performance mode.
This option sets the operator implementation mode by using the built-in configuration file, which is stored in ${INSTALL_DIR}/opp/built-in/op_impl/ai_core/tbe/impl_mode/high_performance.ini.

To ensure compatibility, this argument takes effect only for the operator list in the high_performance.ini file. This list can be used to control the effective scope of operators and ensure that the network models of earlier versions are not affected.
high_precision_for_all: High-precision mode.
This option sets the operator implementation mode by using the built-in configuration file, which is stored in ${INSTALL_DIR}/opp/built-in/op_impl/ai_core/tbe/impl_mode/high_precision_for_all.ini. The list in this file may be updated with the version.

This implementation mode may cause incompatibility. If an operator in the new software package sets the implementation mode (that is, an implementation mode is added for a certain operator in the configuration file), the performance of the earlier network model that uses the high_precision_for_all mode may deteriorate.
high_performance_for_all: High-performance mode.
This option sets the operator implementation mode by using the built-in configuration file, which is stored in ${INSTALL_DIR}/opp/built-in/op_impl/ai_core/tbe/impl_mode/high_performance_for_all.ini. The list in this file may be updated with the version.

This implementation mode may cause incompatibility. If an operator in the new software package sets the implementation mode (that is, an implementation mode is added for a certain operator in the configuration file), the precision of the earlier network model that uses the high_performance_for_all mode may deteriorate.

The preceding implementation modes are distinguished based on the dtype of the operator. Replace ${INSTALL_DIR} with the CANN component directory. For example, if the installation is performed by the root user, the default file storage path is /usr/local/Ascend/cann.

Suggestions and Benefits

--op_select_implmode is for debugging only and is not recommended for setting the operator implementation mode. You are advised to load the .ini configuration file using --op_precision_mode to set the operator precision mode.

If you have higher performance requirements, use high_performance_for_all. If the performance meets requirements, copy the high_performance_for_all.ini file and rename it to ${network_model}.ini. Use different .ini files for different network models. During subsequent model conversion, you can directly use --op_precision_mode to load the saved ${network_model}.ini configuration file.
If you have higher precision requirements, use high_precision_for_all. If the precision meets requirements, copy the high_precision_for_all.ini file and rename it to ${network_model}.ini. Use different .ini files for different network models. During subsequent model conversion, you can directly use --op_precision_mode to load the saved ${network_model}.ini configuration file.
If you use high_performance_for_all and observe significant performance improvement while the precision fails to meet requirements, which is caused by the xxx operator running in high-performance mode, copy the high_performance_for_all.ini file and rename it to ${network_model}.ini. Modify the implementation mode of the xxx operator in this file to the high-precision mode. During subsequent model conversion, you can directly use --op_precision_mode to load the saved ${network_model}.ini configuration file.
If you use high_precision_for_all and observe significant precision improvement while the performance degrades substantially, which is caused by the xxx operator running in high-precision mode, copy the high_precision_for_all.ini file and rename it to ${network_model}.ini. Modify the implementation mode of the xxx operator in this file to the high-performance mode. During subsequent model conversion, you can directly use --op_precision_mode to load the saved ${network_model}.ini configuration file.

The implementation modes of operators in the high_*.ini file are subject to those listed in the all_ops_impl_mode.ini file in ${INSTALL_DIR}/opp/built-in/op_impl/ai_core/tbe/impl_mode.

Replace ${INSTALL_DIR} with the CANN component directory. For example, if the installation is performed by the root user, the default file storage path is /usr/local/Ascend/cann.

Example

--op_select_implmode=high_precision

Restrictions

If you choose the high-performance or high-precision mode for a new operator and do not want to damage the accuracy or performance of the existing networks, you can use either of the following methods to configure the precision mode:
- Using --optypelist_for_implmode to specify the operator to be added
```
--op_select_implmode=high_precision  --optypelist_for_implmode=optype
```
- Using --op_precision_mode to set the operator precision mode
  Build the operator precision mode configuration file op_precision.ini and set the operator precision mode in the file. Set the precision mode of an operator each row. The following is an example:
```
optype1=high_precision
optype2=high_performance
```
  Upload the configured op_precision.ini file to any directory (for example, $HOME/conf) on the server where ATC is located.
```
--op_precision_mode=$HOME/conf/op_precision.ini
```
--op_select_implmode selects between the high-precision mode and high-performance mode for all operators on a network. If an operator supports both high-precision and high-performance, the mode specified by --op_select_implmode is implemented at run time. If an operator supports only high-precision or high-performance, the mode supported is implemented at run time. For example:
If an operator supports only high-precision while --op_select_implmode is set to high_performance, the --op_select_implmode option does not take effect and the high-precision mode is implemented.

Parent topic: Options That Will Be Deprecated in Later Versions