create_quant_config

Applicability

Product	Supported
Atlas A3 training series products/Atlas A3 inference series products	x
Atlas A2 training products/Atlas A2 inference products	x
Atlas 200I/500 A2 inference product	√
Atlas inference series products	√
Atlas training products	√

Description

Finds all quantizable layers in a graph, creates a quantization configuration file, and writes the quantization configuration of the quantizable layers to the configuration file.

Prototype

create_quant_config(config_file, model_file, weights_file, skip_layers=None, batch_num=1, activation_offset=True, config_defination=None)

Parameters

Parameter	Input/Output	Description
config_file	Input	Path (including the file name) of the quantization configuration file. The existing file (if any) in the path will be overwritten upon this API call. A string.
model_file	Input	Definition file (.prototxt) of the Caffe model. A string.
weights_file	Input	Weight file (.caffemodel) of the trained Caffe model. A string.
skip_layers	Input	Layers to skip quantization. Default: None A list of strings. Restrictions: If a simplified quantization configuration file is used as the input, this parameter must be set in the configuration file. In this case, the parameter setting in the input does not take effect.
batch_num	Input	Number of batches taken to generate the quantization factors. An int. Value range: any integer larger than 0. Default: 1 Restrictions: batch_num must not be too large. The product of batch_num and batch_size equals the number of images used during quantization. Too many images consume too much memory. If a simplified quantization configuration file is used as the input, this parameter must be set in the configuration file. In this case, the parameter setting in the input does not take effect.
activation_offset	Input	Whether to quantize activations with offset. Default: True A bool Restrictions: If a simplified quantization configuration file is used as the input, this parameter must be set in the configuration file. In this case, the parameter setting in the input does not take effect.
config_defination	Input	Simplified quantization configuration file quant.cfg generated based on the calibration_config_caffe.proto file. The .proto file is stored in /amct_caffe/proto/* under the AMCT installation directory. For details about the parameters in the .proto file and the generated simplified quantization configuration file quant.cfg, see Simplified PTQ Configuration File. Default: None* A string. Restrictions: If it is set to None, a configuration file is generated based on the remaining arguments (skip_layers, batch_num, and activation_offset). In other cases, a configuration file in JSON format is generated based on config_defination.

Returns

None

Restrictions

Due to the data format changes, the values of quantization parameters in the generated quantization configuration file are different from those in the simplified configuration file. However, the accuracy is not affected.

Example

from amct_caffe import create_quant_config
# Generate a quantization configuration file based on parameters.
create_quant_config(config_file="./configs/config.json",
                    model_file="./pretrained_model/model.prototxt",
                    weights_file="./pretrained_model/model.caffemodel",
                    skip_layers=None,
                    batch_num=1,
                    activation_offset=True)
# Generate a quantization configuration file using a simplified configuration file.
create_quant_config(config_file="./configs/config.json",
                    model_file="./pretrained_model/model.prototxt",
                    weights_file="./pretrained_model/model.caffemodel",
                    config_defination="./configs/quant.cfg")

Flush file:

A quantization configuration file in JSON format. The following is an example. (The quantization configuration file output by this API will be overwritten when quantization is performed again.) For details about the parameters, see Quantization Configuration File.

Uniform quantization configuration file (see IFMR Algorithm for activation quantization)

{
    "version":1,
    "batch_num":2,
    "activation_offset":true,
    "joint_quant":false,
    "do_fusion":true,
    "skip_fusion_layers":[],
    "conv1":{
        "quant_enable":true,
        "activation_quant_params":{
            "max_percentile":0.999999,
            "min_percentile":0.999999,
            "search_range":[
                0.7,
                1.3
            ],
            "search_step":0.01,
            "act_algo":"ifmr",
            "asymmetric":false
        },
        "weight_quant_params":{
            "wts_algo":"arq_quantize",
            "channel_wise":true
        }
    },
    "conv2":{
        "quant_enable":true,
        "activation_quant_params":{
            "max_percentile":0.999999,
            "min_percentile":0.999999,
            "search_range":[
                0.7,
                1.3
            ],
            "search_step":0.01,
            "act_algo":"ifmr",
            "asymmetric":false
        },
        "weight_quant_params":{
            "wts_algo":"arq_quantize",
            "channel_wise":false
        }
     }
}

Uniform quantization configuration file (see HFMG Algorithm for activation quantization)

{
    "version":1,
    "batch_num":2,
    "activation_offset":true,
    "joint_quant":false,
    "do_fusion":true,
    "skip_fusion_layers":[],
    "conv1":{
        "quant_enable":true,
        "activation_quant_params":{
            "act_algo":"hfmg",
            "num_of_bins":4096,
            "asymmetric":false
        },
        "weight_quant_params":{
            "wts_algo":"arq_quantize",
            "channel_wise":true
        }
    }
}

Parent topic: PTQ APIs