AutoRound

AutoRound is a lightweight PTQ algorithm for LLMs capable of learning rounding offsets. Its core mechanism treats rounding offsets as learnable parameters and jointly tunes minimum/maximum scales to enable weights to take optimal rounding directions during low-bit quantization. For details about the algorithm, see Optimize Weight Rounding via Signed Gradient Descent for the Quantization of LLMs.

Note: AutoRound comes with a custom quantize() hook, which is available for PTQ fake quantization. Such hooks are not yet supported in deployment export via export_deploy().