create_quant_config

Function Usage

Finds all quantizable layers in a graph, creates a quantization configuration file, and writes the quantization configuration of the quantizable layers to the file.

Prototype

create_quant_config(config_file, model, input_data, skip_layers=None, batch_num=1, activation_offset=True, config_defination=None)

Command-Line Options

Option	Input/Return	Description	Restriction
config_file	Input	Path and name of the quantization configuration file The existing file (if any) in the path will be overwritten upon this API call.	A string
model	Input	Source model, with weights loaded.	A torch.nn.Module
input_data	Input	Input data of the model. A torch.tensor, equivalent to a tuple(torch.tensor).	A tuple
skip_layers	Input	Layers to skip quantizing.	Default: None A list of strings. Restriction: If a simplified quantization configuration file is used as the input, this parameter must be set in the configuration file. In this case, the parameter setting in the input does not take effect.
batch_num	Input	Number of batches used for quantization, that is, the number of batches used to generate quantization factors.	Type: int Valid Value: an integer greater than or equal to 0 Default value: 1 Restrictions: batch_num must not be too large. The product of batch_num and batch_size equals the number of images used during quantization. Too many images consume too much memory. If a simplified quantization configuration file is used as the input, this parameter must be set in the configuration file. In this case, the parameter setting in the input does not take effect.
activation_offset	Input	Whether to quantize activations with offset.	Default: True A bool. Restriction: If a simplified quantization configuration file is used as the input, this parameter must be set in the configuration file. In this case, the parameter setting in the input does not take effect.
config_defination	Input	Whether to create a simplified quantization configuration file quant.cfg from the calibration_config_pytorch.proto file in /amct_pytorch/proto/calibration_config_pytorch.proto under the AMCT installation path. For details about the parameters in the calibration_config_pytorch.proto file and the generated simplified quantization configuration file quant.cfg, see Simplified PTQ Configuration File.	Default: None A string Restriction: If None, a configuration file is generated based on the remaining arguments (skip_layers, batch_num, and activation_offset). In other cases, a configuration file in JSON format is generated based on this argument.

Return Value

None

Outputs

A quantization configuration file in JSON format. (When quantization is performed again, this API will overwrite the existing configuration file in the output directory.)

{
    "version":1,
    "batch_num":2,
    "activation_offset":true,
    "do_fusion":true,
    "skip_fusion_layers":[],
    "conv1":{
        "quant_enable":true,
        "dmq_balancer_param":0.5,
        "activation_quant_params":{
            "num_bits":8,
            "max_percentile":0.999999,
            "min_percentile":0.999999,
            "search_range":[
                0.7,
                1.3
            ],
            "search_step":0.01,
            "act_algo":"ifmr",
            "asymmetric":false
        },
        "weight_quant_params":{
            "num_bits":8,
            "wts_algo":"arq_quantize",
            "channel_wise":true
        }
    },
    "fc":{
        "quant_enable":true,
        "dmq_balancer_param":0.5,
        "activation_quant_params":{
            "num_bits":8,
            "max_percentile":0.999999,
            "min_percentile":0.999999,
            "search_range":[
                0.7,
                1.3
            ],
            "search_step":0.01,
            "act_algo":"ifmr",
            "asymmetric":false
        },
        "weight_quant_params":{
            "num_bits":8,
            "wts_algo":"arq_quantize",
            "channel_wise":false
        }
    }
}

Examples

import amct_pytorch as amct
# Build a graph of the network to be quantized.
model = build_model()
model.load_state_dict(torch.load(state_dict_path))
input_data = tuple([torch.randn(input_shape)])
model.eval()

# Create a quantization configuration file.
amct.create_quant_config(config_file="./configs/config.json",
                         model=model,
                         input_data=input_data,
                         skip_layers=None,
                         batch_num=1,
                         activation_offset=True)

Parent topic: PTQ APIs