SmoothQuant Algorithm

SmoothQuant is a PTQ method that maintains accuracy while ensuring efficient inference. Since weights are easy to quantize whereas activations are difficult, SmoothQuant introduces a smoothing factor to mitigate activation outliers. Through a mathematically equivalent transformation, the quantization difficulty is shifted from activations to weights. In general, the more activation outliers present, the greater the migration strength required.

For details about the algorithm, click here.