create_quant_config

Applicability

Product	Supported
Atlas A3 training series products/Atlas A3 inference series products	√
Atlas A2 training products/Atlas A2 inference products	√
Atlas 200I/500 A2 inference product	√
Atlas inference series products	√
Atlas training products	√

Description

Finds all quantizable layers in a graph, creates a quantization configuration file, and writes the quantization configuration of the quantizable layers to the configuration file.

Prototype

create_quant_config(config_file, model, input_data, skip_layers=None, batch_num=1, activation_offset=True, config_defination=None)

Parameters

Parameter	Input/Output	Description
config_file	Input	Path (including the file name) of the quantization configuration file. The existing file (if any) in the path will be overwritten upon this API call. A string.
model	Input	Original model for quantization, with weights loaded. A torch.nn.Module.
input_data	Input	Input data of the model. A torch.tensor is replaced with an equivalent tuple(torch.tensor). A tuple.
skip_layers	Input	Layers to skip quantization. Default: None A list of strings. Restrictions: If a simplified quantization configuration file is used as the input, this parameter must be set in the configuration file. In this case, the parameter setting in the input does not take effect.
batch_num	Input	Number of batches taken to generate the quantization factors. An int. Value range: an integer greater than or equal to 0. Default value: 1. Restrictions: batch_num must not be too large. The product of batch_num and batch_size equals the number of images used during quantization. Too many images consume too much memory. If a simplified quantization configuration file is used as the input, this parameter must be set in the configuration file. In this case, the parameter setting in the input does not take effect.
activation_offset	Input	Whether to quantize activations with offset. Default: True A bool. Restrictions: If a simplified quantization configuration file is used as the input, this parameter must be set in the configuration file. In this case, the parameter setting in the input does not take effect.
config_defination	Input	Simplified quantization configuration file quant.cfg generated based on the calibration_config_pytorch.proto file. The .proto file is stored in /amct_pytorch/proto/* under the AMCT installation directory. For details about the parameters in the .proto file and the generated simplified quantization configuration file quant.cfg, see Simplified PTQ Configuration File. Default: None* A string. Restrictions: If it is set to None, a configuration file is generated based on the remaining arguments (skip_layers, batch_num, and activation_offset). In other cases, a configuration file in JSON format is generated based on this argument.

Returns

None

Example

import amct_pytorch as amct
# Build a graph of the network to be quantized.
model = build_model()
model.load_state_dict(torch.load(state_dict_path))
input_data = tuple([torch.randn(input_shape)])
model.eval()

# Create a quantization configuration file.
amct.create_quant_config(config_file="./configs/config.json",
                         model=model,
                         input_data=input_data,
                         skip_layers=None,
                         batch_num=1,
                         activation_offset=True)

Flush file: a quantization configuration file in JSON format. The following is an example. (The quantization configuration file output by this API will be overwritten when quantization is performed again.) For details about the parameters, see Quantization Configuration File.

Uniform quantization configuration file (see IFMR Algorithm for activation quantization)

{
    "version":1,
    "batch_num":2,
    "activation_offset":true,
    "do_fusion":true,
    "skip_fusion_layers":[],
    "conv1":{
        "quant_enable":true,
        "dmq_balancer_param":0.5,
        "activation_quant_params":{
            "num_bits":8,
            "max_percentile":0.999999,
            "min_percentile":0.999999,
            "search_range":[
                0.7,
                1.3
            ],
            "search_step":0.01,
            "act_algo":"ifmr",
            "asymmetric":false
        },
        "weight_quant_params":{
            "num_bits":8,
            "wts_algo":"arq_quantize",
            "channel_wise":true
        }
    },
    "fc":{
        "quant_enable":true,
        "dmq_balancer_param":0.5,
        "activation_quant_params":{
            "num_bits":8,
            "max_percentile":0.999999,
            "min_percentile":0.999999,
            "search_range":[
                0.7,
                1.3
            ],
            "search_step":0.01,
            "act_algo":"ifmr",
            "asymmetric":false
        },
        "weight_quant_params":{
            "num_bits":8,
            "wts_algo":"arq_quantize",
            "channel_wise":false
        }
    }
}

Uniform quantization configuration file (see HFMG Algorithm for activation quantization)

{
    "version":1,
    "batch_num":2,
    "activation_offset":true,
    "do_fusion":true,
    "skip_fusion_layers":[],
    "conv1":{
        "quant_enable":true,
        "dmq_balancer_param":0.5,
        "activation_quant_params":{
            "num_bits":8,
            "act_algo":"hfmg",
            "num_of_bins":4096,
            "asymmetric":false
        },
        "weight_quant_params":{
            "num_bits":8,
            "wts_algo":"arq_quantize",
            "channel_wise":true
        }
    }
}

Simplified configuration file for adaptive rounding quantization (see ADA Algorithm for weight quantization)

"layer_name1":{
        "quant_enable":true,
        "weight_quant_params":{
            "wts_algo":"ada_quantize",
            "num_iteration":10000,
            "reg_param":0.1,
            "beta_range":[20,2], 
		"warm_start":0.2,
		"num_bits":8,
		"channel_wise":true
        }
    }

Parent topic: PTQ APIs