ptq

Feature Overview

• Feature entry: $HOME/.local/lib/python3.x/site-packages/amct_pytorch/cli/llm/ptq.py
• Description: This entry performs PTQ optimization training. The workflow is as follows:
  1. Data loading: Loads the calibration dataset generated via extract_ptq_data.
  2. Layer-wise optimization: Performs layer-by-layer quantization optimization for units specified by --quant_target (quantization target).
  3. Loss minimization: Takes the outputs from the original unquantized model as the ground truth (GT), and optimizes quantization parameters by minimizing the mean squared error (MSE) between the outputs of quantized modules and the GT.
  4. Parameter export: Automatically exports PTQ parameters for each unit after the optimization completes.
• Output: Generates layer_*.pt files in the PTQ parameter directory.
• Constraints:
  • The quant_target parameter must specify only one quantization target unit.
  • Currently, only granularity=block is supported. PTQ logic for the model granularity is reserved for future use.
  • The value of --granularity must be the same as that of the corresponding parameter in extract_ptq_data.

Instantiation Operation: PTQ Parameter Training

PTQ requires processing only one quantization target (--quant_target) at a time and supports quantization training at block granularity. After the training completes, PTQ quantization parameters are saved to the corresponding parameter directory. The command is as follows:

1

python3 -m amct_pytorch.ptq --model /path/to/model --model_name qwen3 --device npu:0 --granularity block --quant_target mlp --quant_dtype int --bit_config amct_pytorch/configs/example_w4a8.yaml --algos lwc lac --data_dir ./outputs/qwen3_ptq_data --start_block_idx 0 --end_block_idx 32 --epochs 15 --base_lr 1e-5 --optimizer adamw --output_dir ./outputs/qwen3_ptq

For details about the parameters, see Parameters.