文档

--op_select_implmode

Description

The function of this parameter does not evolve and will be deprecated in later versions. You are advised to use --op_precision_mode.

Selects an operator implementation mode. Certain operators built in the Ascend AI Processor can be implemented in either high-precision or high-performance mode at model build time.

In high-precision mode, Taylor's theorem or Newton's method is used to improve operator accuracy with float16 input. In high-performance mode, the optimal performance is implemented without affecting the network precision (float16).

See Also

None

Argument

  • high_precision: High-precision implementation mode.

    This option sets the operator implementation mode by using the built-in configuration file, which is stored in ${INSTALL_DIR}/opp/op_impl/built-in/ai_core/tbe/impl_mode/high_precision.ini.

    To ensure compatibility, this argument takes effect only for the operator list in the high_precision.ini file. This list can be used to control the effective scope of operators and ensure that the network models of earlier versions are not affected.

  • high_performance (default): High-performance implementation mode.

    This option sets the operator implementation mode by using the built-in configuration file, which is stored in ${INSTALL_DIR}/opp/built-in/op_impl/ai_core/tbe/impl_mode/high_performance.ini.

    To ensure compatibility, this argument takes effect only for the operator list in the high_performance.ini file. This list can be used to control the effective scope of operators and ensure that the network models of earlier versions are not affected.

  • high_precision_for_all: High-precision mode.

    This option sets the operator implementation mode by using the built-in configuration file, which is stored in ${INSTALL_DIR}/opp/built-in/op_impl/ai_core/tbe/impl_mode/high_precision_for_all.ini. The list in this file may be updated with the version.

    This implementation mode may cause incompatibility. If an operator in the new software package sets the implementation mode (that is, an implementation mode is added for a certain operator in the configuration file), the performance of the earlier network model that uses the high_precision_for_all mode may deteriorate.

  • high_performance_for_all: High-performance mode.

    This option sets the operator implementation mode by using the built-in configuration file, which is stored in ${INSTALL_DIR}/opp/built-in/op_impl/ai_core/tbe/impl_mode/high_performance_for_all.ini. The list in this file may be updated with the version.

    This implementation mode may cause incompatibility. If an operator in the new software package sets the implementation mode (that is, an implementation mode is added for a certain operator in the configuration file), the precision of the earlier network model that uses the high_performance_for_all mode may deteriorate.

The preceding implementation modes are distinguished based on the dtype of the operator. Replace ${INSTALL_DIR} with the actual CANN component directory. If the Ascend-CANN-Toolkit package is installed as the root user, the CANN component directory is /usr/local/Ascend/ascend-toolkit/latest.

Suggestions and Benefits

--op_select_implmode is for debugging only and is not recommended for setting the operator implementation mode. You are advised to load the .ini configuration file by setting --op_precision_mode instead.

  • If you have higher performance requirements, you are advised to use high_performance_for_all. If the performance meets the requirements, you are advised to copy the high_performance_for_all.ini file and rename it as network model.ini. Use different .ini files for different network models. During subsequent model conversion, you can directly use --op_precision_mode to load the saved network model.ini configuration file.
  • If you have higher precision requirements, you are advised to use high_precision_for_all. If the precision meets the requirements, you are advised to copy the high_precision_for_all.ini file and rename it as network model.ini. Use different .ini files for different network models. During subsequent model conversion, you can directly use --op_precision_mode to load the saved network model.ini configuration file.
  • If the performance is greatly improved but the accuracy does not meet the requirements when you use high_performance_for_all because the xxx operator uses the high-performance mode, copy the high_performance_for_all.ini file and rename it as network model.ini. In addition, change the implementation mode of the xxx operator in the file to the high-precision mode. During subsequent model conversion, you can directly use --op_precision_mode to load the saved network model.ini configuration file.
  • If the precision is greatly improved but the performance deteriorates greatly when you use high_precision_for_all because the xxx operator uses the high-precision mode, copy the high_precision_for_all.ini file and rename it as network model.ini. In addition, change the implementation mode of the xxx operator in the file to the high-performance mode. During subsequent model conversion, you can directly use --op_precision_mode to load the saved network model.ini configuration file.

The implementation modes of operators in the high_*.ini file are subject to those listed in the all_ops_impl_mode.ini file in ${INSTALL_DIR}/opp/built-in/op_impl/ai_core/tbe/impl_mode.

Replace ${INSTALL_DIR} with the actual CANN component directory. If the Ascend-CANN-Toolkit package is installed as the root user, the CANN component directory is /usr/local/Ascend/ascend-toolkit/latest.

Example

--op_select_implmode=high_precision

Applicability

Atlas 200/300/500 Inference Product

Atlas Training Series Product

Restrictions

  • If you choose the high-performance or high-precision mode for a new operator and do not want to damage the accuracy or performance of the existing networks, you can use either of the following methods to configure the precision mode:
    • Using --optypelist_for_implmode to specify the operator to be added
      --op_select_implmode=high_precision  --optypelist_for_implmode=optype
    • Using --op_precision_mode to set the operator precision mode

      Build the operator precision mode configuration file op_precision.ini and set the operator precision mode in the file. Set the precision mode of an operator each row. The following is an example:

      optype1=high_precision
optype2=high_performance

      Upload the configured op_precision.ini file to any directory (for example, $HOME/conf) on the server where ATC is located.

      --op_precision_mode=$HOME/conf/op_precision.ini
  • --op_select_implmode selects between the high-precision mode and high-performance mode for all operators on a network. If an operator supports both high-precision and high-performance, the mode specified by --op_select_implmode is implemented at run time. If an operator supports only high-precision or high-performance, the mode supported is implemented at run time. For example:

    If an operator supports only high-precision while --op_select_implmode is set to high_performance, the --op_select_implmode option does not take effect and the high-precision mode is implemented.

Parent topic: Operator Tuning Options