Overview

This section describes graph-based quantization, including the quantization types, quantization processes, and API call examples.

Based on whether retraining is required, quantization is classified into PTQ and QAT. For details about quantization concepts, see Quantization. For details about quantization algorithms, see Compression Algorithms.

• PTQ

  Quantization takes two forms: Manual Quantization and Accuracy-based Automatic Quantization based on whether the quantization configuration file is manually tuned after quantization.

  If the quantized model fails to meet the accuracy requirements, perform accuracy-aware automatic quantization or manual tuning. Accuracy-aware automatic quantization is recommended.

• Quantization aware training

  Currently, QAT supports quantization only for float32 network models.

  Only manual quantization is supported at present. If the quantized model fails to meet the accuracy requirements, perform manual tuning.