--keep_dtype

Applicability

Product	Supported
Atlas A3 training products/Atlas A3 inference products	√
Atlas A2 training products/Atlas A2 inference products	√
Atlas 200I/500 A2 inference products	√
Atlas inference products	√
Atlas training products	√

Description

Keeps the computation precision of some operators unchanged during the compilation of the original network model.

In the inference scenario, --precision_mode can be used to set the global precision mode of a network model, but it may result in performance or accuracy problems on particular operators. Therefore, you can use --keep_dtype to keep the precision of these operators unchanged at build time.

Arguments

Argument: directory (including the file name) of the operator configuration file, which lists operator names or operator types. Each operator must be in a separate line.

Restrictions:

To specify optypes, follow the OpType::typeName format. Put each optype in a separate line. Only optypes defined by Ascend IRs are supported. For details about how to view the optype, see How Do I Determine the Mapping Between Operators in the Original Network Model and Operators Supported by Ascend AI Processor?.
To specify opnames, follow the Opname format. Put each opname in a separate line. Set the value of Opname to the operator name of the original model.

Format: The directory (including the file name) can contain letters, digits, underscores (_), hyphens (-), periods (.), colons (:), and Chinese characters.

Suggestions and Benefits

None

Examples

To specify opnames, use the following template (named execeptionlist.cfg).
```
Opname1
Opname2
...
```
To specify optypes, use the following template (named execeptionlist.cfg).
```
OpType::TypeName1
OpType::TypeName2
...
```

The Relu operator in the TensorFlow ResNet50 network model is used as an example. The operator type defined by Ascend IR is Relu.

# Set the operator name.
fp32_vars/Relu
# Set the operator type.
OpType::Relu

Upload your execeptionlist.cfg file to any directory (for example, ${HOME}) on the server where AOE is located.

--keep_dtype=${HOME}/execeptionlist.cfg --precision_mode=force_fp16

During model compilation, the precision of the operators specified in the execeptionlist.cfg file will be preserved while those not specified in this file will be processed according to --precision_mode.

Dependencies and Restrictions