Quantile

The Quantile algorithm takes the numerical-distribution characteristics of the model into account. It dynamically adjusts the quantization range via quantile statistics to mitigate the impact of extreme values on quantization accuracy. The algorithm workflow is as follows:

Take calibration data of 512 batches as an example:

1. Initialization: Record the maximum value of the first batch as the initial quantization threshold.
2. Moving average: Perform exponential moving average on the maximum values of subsequent batches to smooth the impact of extreme values.
3. Scaling factor calculation: Calculate quantization scaling factors based on the final statistical values.

Data representation:

This algorithm applies to scenarios requiring high quantization accuracy, where the data distribution contains outliers or anomalies and calibration data is available. Its advantages are as follows:

• Higher accuracy than the Cast algorithm
• Low resource overhead for quantization calibration
• Effective suppression of the impact of extreme values