Parameters

--model

deepseek-ai/DeepSeek-V4-Pro

Path to model weights or model identifier. During export for deployment, the value must be a local model directory.

--model_name

deepseek-ai/DeepSeek-V4-Pro

Name of the internal AMCT model adapter, which must match the registered model, such as qwen3 or deepseek_v4.

--device

npu:0

Running device. NPU or GPU is recommended to accelerate computation.

--granularity

model

Working granularity, that is, the unit of quantization or processing. Available options:

block: processing by block (finer-grained). This mode may deliver better performance but consumes more time.

model: processing the entire model as a whole (coarse-grained and fast). This mode is not supported for some workflows.

--seed

0

Random seed, which is used to control the consistency of random behaviors (such as data sampling and initialization) in experiments. Setting it to a fixed value ensures that the results can be reproduced.

--quant_target

[]

Quantization target. The following combinations are supported:

mlp: multi-layer perceptron.

moe: mixture of experts.

attn-linear: attention linear layer.

attn-cache: attention cache.

--seq_len

4096

Input sequence length used for calibration and evaluation, which can be 1024, 2048, or 4096. It determines the text length for model processing, which is critical during quantization calibration.

--data_dir

Empty string

Directory for storing or reading PTQ intermediate data.

--output_dir

./outputs

Output directory, which stores all generated content, including log files, PTQ parameter configurations, and the final deployable model.

--eval_mode

bf16

Evaluation mode that specifies the precision format used by the model to compute the PPL. Supported options are as follows:

bf16: BF16, a half-precision floating-point format commonly used to accelerate inference while maintaining high accuracy.

quant: quantization mode, which may refer to quantization formats such as INT8/INT4, used to reduce memory and computation overhead at the cost of minor accuracy degradation.

--wikitext_final_out

Empty string

Path to the final WikiText output file after dataset processing. This parameter is reserved for future use.

--nsamples

128

Number of calibration samples. extract_ptq_data loads this number of samples from Pileval.

--quant_dtype

Left empty

Data type of quantized values, which can be mxfp or int currently. This parameter is mandatory when eval_mode is set to quant.

--bit_config

None

Path to the YAML-formatted configuration file that defines bit-width policies. If this parameter is not provided, the default W16A16 configuration is applied. For common configurations, see the table below.

--algos

[]

List of quantization algorithms to enable. Algorithms are automatically applied to weights, activations, or structures based on their registered targets.

--is_per_tensor

False

Whether to use per-tensor statistics to determine the clipping range for the learned activation clipping (LAC) algorithm.

--k_size

128

Size of the learnable Hadamard-like structure transformation matrix.

bf16.yaml

All layers remain in 16-bit precision.

Left empty.

No configuration is required; 16-bit is used by default.

w8a8.yaml

Global W8A8 (8-bit weights and 8-bit activations).

w4a8.yaml

Global W4A8 (8-bit weights and 4-bit activations).

w4a4.yaml

Global W4A4 (4-bit weights and 4-bit activations).

example_w4a8.yaml

Example of a custom quantization solution.

--cali_bsz

4

Batch size used for PTQ calibration.

During PTQ, a small subset of data (calibration dataset) is used to tune quantization parameters such as scale factors and zero points. This parameter defines the number of samples processed per calibration iteration.

--base_lr

1e-5

Base learning rate, which determines the step size for updating model parameters in each iteration.

--optimizer

adamw

Optimizer that specifies the algorithm used to update model parameters. Supported optimizers:

adamw: improved version of Adam that generally handles weight decay better.

adam: classic Adam optimizer.

sgd: stochastic gradient descent.

cayley: less common optimizer.

--weight_decay

0.0

Weight decay, applicable to certain optimizers. A regularization technique that adds a penalty term proportional to the weight magnitude to the loss function to prevent model overfitting. The value 0.0 indicates that weight decay is not used during this optimization process.

--momentum

0.9

SGD momentum.

When the SGD optimizer is used, momentum accelerates convergence and helps escape local optima by updating parameters with the exponentially weighted moving average of past gradients.

--lr_scheduler

cosine

Learning rate scheduler that defines how to dynamically adjust the learning rate during training. Supported options:

• none: learning rate unchanged.
• cosine: Cosine annealing, where the learning rate smoothly decreases following a cosine curve.
• step: step-wise decay, where the learning rate is multiplied by a coefficient at fixed-step intervals.

The default value cosine delivers smooth learning-rate decay and facilitates better model convergence.

--min_lr

0.0

Minimum learning rate. This parameter is not directly used in common schedulers in the current version.

It sets a lower bound for the learning rate when certain schedulers such as cosine or step are applied. The learning rate will not drop below this threshold even if the scheduler requests further reduction.

--lr_step_size

1

Step size for StepLR.

This parameter takes effect only when --lr_scheduler is set to step. It defines the interval (in epochs) at which the learning rate is decayed.

--lr_gamma

0.1

Decay coefficient for StepLR.

This parameter is valid only when --lr_scheduler is set to step. It defines the multiplier applied to the learning rate upon each decay step.

--epochs

15

Number of PTQ optimization epochs, specifying total rounds for calibration and optimization. One epoch means that the entire calibration dataset is traversed once.

--start_block_idx

0

Starting block index (inclusive).

This parameter specifies the model block from which optimization begins.

--end_block_idx

61

Ending block index (exclusive). It specifies the last block for optimization.

--attn_linear_param_dir

Empty string

Directory for PTQ parameters of attention linear layers.

--attn_cache_param_dir

Empty string

Directory for PTQ parameters of attention cache components.

--moe_mlp_param_dir

Empty string

Directory for PTQ parameters of MoE or MLP components.