--customize_dtypes
Description
Customizes the computing precision of one or more operators during model building.
See Also
- If this option is used together with --precision_mode or --precision_mode_v2, other operators (except the operator specified by this option) in the model are built based on the precision mode specified by --precision_mode or --precision_mode_v2.
- Both --customize_dtypes and --keep_dtype can be used to set the computation precision of operators. If the model inference precision needs to be improved, you are advised to use --keep_dtype to maintain the precision of the original graph. If the precision still cannot be improved, you can use --customize_dtypes to customize the computation precision of one or more operators.
Note that if --customize_dtypes is used and the operator name is configured, the operator name may change due to operations such as fusion and splitting during internal model tuning. As a result, the configuration does not take effect and the precision is not improved. In this case, you need to obtain logs to locate the fault. For details about logs, see Log Reference.
- If both --customize_dtypes and --keep_dtype are used, the precision set by --customize_dtypes takes precedence.
Argument
Argument: Directory (including the file name) of the operator configuration file, which lists custom operator names or operator types. Each operator must be in a separate line.
Restrictions:
- To specify operator names, follow the Opname::InputDtype:dtype1,...,OutputDtype:dtype1,... format. Put each operator name in a separate line. dtype1, dtype2, and more data types must correspond to the inputs and outputs of the custom operators.
- To specify operator types, follow the OpType::TypeName:InputDtype:dtype1,...,OutputDtype:dtype1,... format. Put each operator type in a separate line. dtype1, dtype2, and more data types must correspond to the inputs and outputs of the custom operators. The operator type must be OpType of the Ascend IR–defined operator. For details about how to view the operator type, see How Do I Determine the Mapping Between Operators in the Original Network Model and Operators Supported by Ascend AI Processors?.
- For the same operator, if both Opname and OpType are configured, the Opname configuration is used during building.
- The computing precision of an operator specified by this option does not take effect if the operator is fused during model conversion.
Format: The path (including the file name) can contain letters, digits, underscores (_), hyphens (-), periods (.), colons (:), and Chinese characters.
Suggestions and Benefits
None
Example
- To specify opnames, use the following template (named customize_dtypes.cfg).
Opname1::InputDtype:dtype1,dtype2,...,OutputDtype:dtype1,... Opname2::InputDtype:dtype1,dtype2,...,OutputDtype:dtype1,...
- To specify optypes, use the following template (named customize_dtypes.cfg).
OpType::TypeName1:InputDtype:dtype1,dtype2,...,OutputDtype:dtype1,... OpType::TypeName2:InputDtype:dtype1,dtype2,...,OutputDtype:dtype1,...
For details about the computing precisions supported by the operator, see Operator Acceleration Library API Reference.
The Relu operator in the TensorFlow ResNet50 network model is used as an example. The operator type defined by Ascend IR is Relu. This operator has only one input and one output. The configuration example is as follows:
- Set the operator name.
fp32_vars/Relu::InputDtype:float16,OutputDtype:int8
- Set the operator type.
OpType::Relu:InputDtype:float16,OutputDtype:int8
Upload your customize_dtypes.cfg file to any directory (for example, $HOME) on the server where ATC is located.
--customize_dtypes=$HOME/customize_dtypes.cfg --precision_mode=force_fp16
During model compilation, the computing precision of the operators specified in the customize_dtypes.cfg file will be preserved while those not specified in this file will be processed according to --precision_mode or --precision_mode_v2.
Applicability
Restrictions
- This option takes precedence over --precision_mode/--precision_mode_v2 and --keep_dtype, which may cause inference precision or performance degradation.
- If an operator does not support the computing precision specified by this option, the model building fails.