Documentation

Setting the Precision Mode

Computations are done in float16 unless otherwise specified.

The high-precision mode can improve the inference accuracy but might affect the inference performance. To improve the accuracy while preserving the performance, preserve the precision of selected operators.

Setting the Model to High-Precision Mode

  1. Include the --precision_mode option in your atc command line.

    Set this option as follows to preserve float32 calculation for operators supporting float32 and force float16 calculation for operators not supporting float32.

    --precision_mode=allow_fp32_to_fp16

    For details about this parameter, see --precision_mode in ATC Instructions.

  2. Run inference on the new OM.

Preserving the Precision of Selected Operators

  1. Include the --keep_dtype and --precision_mode options in your atc command line.

    Example:

    --keep_dtype=$HOME/execeptionlist.cfg --precision_mode=force_fp16

    Upload the configuration file (for example, execeptionlist.cfg) to any directory on the server installed with the ATC tool. The following is an example of the file content. Put the operator names in separate lines.

    Opname1 
Opname2 
...

    For details about this parameter, see --keep_dtype in ATC Instructions.

  2. Run inference on the new OM.