--customize_dtypes

Applicability

Description

Customizes the computing precision of one or more operators during model compilation.

See Also

• Use this option in conjunction with --precision_mode.
• If both --customize_dtypes and --keep_dtype are used, the precision set by --customize_dtypes takes precedence.

Arguments

Argument: directory (including the name) of the operator configuration file, which lists custom operator names or operator types. Each operator must be in a separate line.

Restrictions:

• To specify opnames, follow the Opname::InputDtype:dtype1,...,OutputDtype:dtype1,... format. Put each opname in a separate line. dtype1, dtype2, and more data types must correspond to the inputs and outputs of the custom operators. Set the value of Opname to the operator name of the original model.
• To specify optypes, follow the OpType::TypeName:InputDtype:dtype1,...,OutputDtype:dtype1,... format. Put each optype in a separate line. dtype1, dtype2, and more data types must correspond to the inputs and outputs of the custom operators. The operator type must be OpType of the Ascend IR–defined operator. For details about how to view the operator type, see How Do I Determine the Mapping Between Operators in the Original Network Model and Operators Supported by AI processor?.
• For the same operator, if both Opname and OpType are configured, the Opname configuration is used during building.
• The computing precision of an operator specified by this option does not take effect if the operator is fused during model conversion.

Format: The directory (including the file name) can contain uppercase and lowercase letters (A–Z, a–z), digits (0–9), underscores (_), hyphens (-), periods (.), colons (:), and Chinese characters.

Suggestions and Benefits

None

Examples

• To specify opnames, use the following template (named customize_dtypes.cfg).

  Opname1::InputDtype:dtype1,dtype2,...,OutputDtype:dtype1,...
  Opname2::InputDtype:dtype1,dtype2,...,OutputDtype:dtype1,...

• To specify optypes, use the following template (named customize_dtypes.cfg).

  OpType::TypeName1:InputDtype:dtype1,dtype2,...,OutputDtype:dtype1,...
  OpType::TypeName2:InputDtype:dtype1,dtype2,...,OutputDtype:dtype1,...

For details about the computing precisions supported by the operator, see Operator Library.

The Relu operator in the TensorFlow ResNet50 network model is used as an example. The operator type defined by Ascend IR is Relu. This operator has only one input and one output. The configuration examples are as follows:

• Set the operator name.

  fp32_vars/Relu::InputDtype:float16,OutputDtype:int8

• Set the operator type.

  OpType::Relu:InputDtype:float16,OutputDtype:int8

Upload your customize_dtypes.cfg file to any directory (for example, ${HOME}) on the server where AOE is located.

--customize_dtypes=${HOME}/customize_dtypes.cfg --precision_mode=force_fp16

During model compilation, the computing precision of the operators specified in the customize_dtypes.cfg file will be preserved while those not specified in this file will be processed according to --precision_mode.

Restrictions

1. This option takes precedence over --precision_mode and --keep_dtype, which may cause inference accuracy or performance degradation.
2. If an operator does not support the computing precision specified by this option, the model building fails.