--keep_dtype

Description

Keeps the computation precision of some operators unchanged during the build of the original network model.

In the inference scenario, --precision_mode or --precision_mode_v2 can be used to set the global precision mode of a network model, but it may result in performance or accuracy problems on particular operators. Therefore, you can use --keep_dtype to keep the computation precision of these operators unchanged during the build of the original network model. If the computation precision is not supported by the Ascend AI Processor, the system automatically uses the high precision supported by the operators for computation.

See Also

• Use this option in conjunction with --precision_mode or --precision_mode_v2. However, --keep_dtype does not take effect when --precision_mode is set to must_keep_origin_dtype or --precision_mode_v2 is set to origin.
• Both --customize_dtypes and --keep_dtype can be used to set the computation precision of operators. If the model inference precision needs to be improved, you are advised to use --keep_dtype to maintain the precision of the original graph. If the precision still cannot be improved, you can use --customize_dtypes to customize the computation precision of one or more operators.

  Note that if --customize_dtypes is used and the operator name is configured, the operator name may change due to operations such as fusion and splitting during internal model tuning. As a result, the configuration does not take effect and the precision is not improved. In this case, you need to obtain logs to locate the fault. For details about logs, see Log Reference.

• If both --customize_dtypes and --keep_dtype are used, the precision set by --customize_dtypes takes precedence.

Argument

Argument: Directory (including the file name) of the operator configuration file, which lists operator names or operator types. Each operator must be in a separate line.

Restrictions: To specify optypes, follow the OpType::typeName format. Put each optype in a separate line. Only optypes defined by Ascend IRs are supported. For details about how to view the optype, see How Do I Determine the Mapping Between Operators in the Original Network Model and Operators Supported by Ascend AI Processors?.

Format: The directory (including the file name) can contain letters, digits, underscores (_), hyphens (-), periods (.), and Chinese characters.

Suggestions and Benefits

None

Example

• To specify opnames, use the following template (named execeptionlist.cfg).

  Opname1
  Opname2
  ...

• To specify optypes, use the following template (named execeptionlist.cfg).

  OpType::TypeName1
  OpType::TypeName2
  ...

The Relu operator in the TensorFlow ResNet50 network model is used as an example. The operator type defined by Ascend IR is Relu.

# Set the operator name.
fp32_vars/Relu
# Set the operator type.
OpType::Relu

Upload your execeptionlist.cfg file to any directory (for example, $HOME) on the server where ATC is located.

--keep_dtype=$HOME/execeptionlist.cfg --precision_mode=force_fp16

At build time, the precision of the operators specified in the execeptionlist.cfg file will be preserved while those not specified in this file will be processed according to --precision_mode or --precision_mode_v2.

Applicability

Atlas 200/300/500 Inference Product

Atlas Training Series Product

Dependencies and Restrictions

None