GPTQ

Generative Pre-trained Transformer Quantization (GPTQ) is a weight-only PTQ method designed for LLMs. Its core idea is layer-wise greedy quantization, which minimizes quantization error through mathematical optimization. For each quantizable operator, the Hessian matrix is computed based on activations, and weights are adjusted block-wise to minimize the quantization error. The final outputs are the adjusted weights and quantization factors.

For details about the algorithm, click here.