extract_ptq_data

Feature Overview

• Feature entry: $HOME/.local/lib/python3.x/site-packages/amct_pytorch/cli/llm/extract_ptq_data.py
• Description: This entry extracts PTQ calibration data. The workflow is as follows:
  1. Calibration data loading: Loads the Pileval calibration samples.
  2. Inference execution: Performs embedding encoding and block-by-block forward propagation.
  3. Intermediate data collection: Captures intermediate data such as the input and keyword arguments (kwargs) required for PTQ at the specified hook position for subsequent PTQ workflows.
• Output: Outputs PTQ data including block- or unit-level inputs and kwargs under the directory specified by --data_dir.
• Constraints:
  • The --quant_target parameter must specify only one target.
  • The value of --granularity must be the same as that of the corresponding parameter in ptq.

Instantiation Operation: PTQ Data Extraction

In the block-based PTQ data extraction workflow, the system automatically locates target layers and registers hooks according to the --quant_target configuration to capture activations or weights. Data capture positions for different quantization targets are described below:

• Attention modules (attn-linear and attn-cache): Data is captured after the attention norm.
• FFN/MoE modules (mlp and moe): Data is captured after the FFN norm.

Command:

1

python3 -m amct_pytorch.extract_ptq_data --model /path/to/model --model_name qwen3 --device npu:0 --quant_target mlp --seq_len 4096 --data_dir ./outputs/qwen3_ptq_data

For details about the parameters, see Parameters.