--op_precision_mode

Description

Sets the precision mode of one or more specified operators during internal processing.

See Also

• This option is mutually exclusive with --op_select_implmode and --optypelist_for_implmode. If they are used together, only the mode specified by --op_precision_mode takes effect.

Figure 1 shows the associated options.

To set the operator precision mode:

1. Obtain the --op_precision_mode option and check whether the .ini configuration file exists. If the file exists, parse the file and read the operator precision mode. Otherwise, an error is reported.
2. If --op_precision_mode does not exist, read --op_select_implmode.
   1. Check whether this option is set to high_xxx_for_all. If yes, parse the high_xxx_for_all.ini file and read the operator precision mode.
   2. If the high_xxx option is configured, check whether --optypelist_for_implmode is configured. If yes, read the operator precision mode specified by this option. If no, parse the high_xxx.ini file and read the operator precision mode.

Argument

Argument: Directory (including the file name) of the .ini configuration file. The following precision mode is supported:

• high_precision
• high_performance
• support_out_of_bound_index: indicates that the out-of-bounds verification is performed on the indices of the gather, scatter, and segment operators. The verification deteriorates the operator execution performance.
• keep_fp16: The FP16 data type is used for internal processing of operators. In this scenario, the FP16 data type is not automatically converted to the FP32 data type. If the performance of FP32 computation does not meet the expectation and high precision is not required, you can select the keep_fp16 mode. This low-precision mode sacrifices the precision for improving the performance, which is not recommended.
• super_performance: indicates ultra-high performance. Compared with high performance, the algorithm calculation formula is optimized.

You can view the precision or performance mode supported by an operator in the opp/built-in/op_impl/ai_core/tbe/impl_mode/all_ops_impl_mode.ini file in the file storage path with the CANN software installed.

Format: The directory (including the file name) can contain letters, digits, underscores (_), hyphens (-), periods (.), and Chinese characters.

Restrictions:

• Currently, the operator precision can be set only by using the .ini configuration file. The configuration file content is formatted as operator type=precision mode (key-value pairs). The precision mode of an operator is set in each row.
• The operator type must be OpType of the Ascend IR–defined operator. For details about how to view the operator type, see How Do I Determine the Mapping Between Operators in the Original Network Model and Operators Supported by Ascend AI Processors?.

Suggestions and Benefits

• You are advised not to set this option. It is used if you need to adjust the precision of a specific operator using the .ini configuration file in the case that you fail to obtain optimal network performance or accuracy in high-performance or high-precision mode.
• You are advised to load the .ini configuration file saved by using the --op_select_implmode option. For details, see Suggestions and Benefits.

Example

Create the op_precision.ini configuration file to set operator precision modes. Set a precision mode by operator type (low priority) or by node name (high priority) in each line of the file.

The configuration example is as follows:

[ByOpType]
optype1=high_precision
optype2=high_performance
optype4=support_out_of_bound_index

[ByNodeName]
nodename1=high_precision
nodename2=high_performance
nodename4=support_out_of_bound_index

Upload the configured op_precision.ini file to any directory (for example, $HOME/conf) on the server where ATC is located.

--op_precision_mode=$HOME/conf/op_precision.ini

Applicability

Atlas 200/300/500 Inference Product

Atlas Training Series Product

Dependencies and Restrictions

None