Manual Tuning

If the QAT accuracy does not meet the requirements, you can manually adjust the parameters in the config.json file. This section provides the adjustment principles and parameter description.

Tuning Workflow

If you find that the accuracy of the model quantized based on the initial config.json file generated by the create_quant_retrain_config API call is not as expected, you can tune the PTQ configuration parameters until the accuracy meets your requirement.

Run quantization based on the initial config.json file generated by the create_quant_retrain_config API call. If the accuracy of the quantized model is satisfactory, stop tuning the configuration parameters. Otherwise, go to 2.

Cancel quantization at some quantization layers, that is, change the value of retrain_enable to false. Generally, the input and output layers are likely to be quantization-sensitive. Skip quantizing the input and output layers first.

If you have desirable settings for clip_max and clip_min, modify the quantization configuration file as follows.

{
    "version":1,
    "layername1":{
        "retrain_enable":true,
        "retrain_data_config":{
            "algo":"ulq_quantize",
            "clip_max":3.0,
            "clip_min":-3.0
        },
        "retrain_weight_config":{
            "algo":"arq_retrain",
            "channel_wise":true
        }
    },
    "layername2":{
        "retrain_enable":true,
        "retrain_data_config":{
            "algo":"ulq_quantize",
            "clip_max":3.0,
            "clip_min":-3.0
        },
        "retrain_weight_config":{
            "algo":"arq_retrain",
            "channel_wise":true
        }
    }
}

Run quantization based on the new configuration. If the accuracy of the quantized model is satisfactory, stop tuning the configuration parameters. Otherwise, it indicates that your model is not suitable for QAT and the QAT configuration should be removed.

Quantization Configuration File

The following is an example of the config.json quantization aware training configuration file generated by calling the create_quant_retrain_config API. (Ensure that the layer name is unique when modifying the JSON file.)

{
    "version":1,
    "conv1":{
        "retrain_enable":true,
        "retrain_data_config":{
            "algo":"ulq_quantize"
        },
        "retrain_weight_config":{
            "algo":"arq_retrain",
            "channel_wise":true
        }
    },
    "conv2_1/expand":{
        "retrain_enable":true,
        "retrain_data_config":{
            "algo":"ulq_quantize"
        },
        "retrain_weight_config":{
            "algo":"arq_retrain",
            "channel_wise":true
        }
    },
    "conv2_1/dwise":{
        "retrain_enable":true,
        "retrain_data_config":{
            "algo":"ulq_quantize"
        },
        "retrain_weight_config":{
            "algo":"arq_retrain",
            "channel_wise":true
        }
    },
}

Command-Line Options

The following describes the configuration parameters available in the configuration file. Note that Table 7 to Table 9 are available only when you manually tune the quantization configuration file.

**Table 1** version
Description	Version number of the quantization configuration file
Type	int
Value	1
Command-Line Options	Currently, only version 1 is available.
Recommended Value	1
Required/Optional	This function is optional.

**Table 2** retrain_enable
Description	QAT enable
Type	bool
Value	true or false
Parameter Description	true: on false: off
Recommended	true
Required/Optional	This function is optional.

**Table 3** retrain_data_config
Description	Activation quantization configuration
Type	object
Value	-
Command-line options	Includes the following parameters: algo: quantization algorithm select, defaulted to ulq_quantize. clip_max: upper bound of clipping-based quantization, defaulted to be empty. clip_min: lower bound of clipping-based quantization, defaulted to be empty. fixed_min: whether to fix the minimum value of clipping-based quantization to 0, defaulted to be empty.
Recommended Value	-
Required/Optional	Optional

**Table 4** retrain_weight_config
Description	Weight quantization configuration
Type	object
Value	-
Command-line options	Includes the following parameters: algo: quantization algorithm select, defaulted to arq_retrain. channel_wise
Recommended Value	-
Required/Optional	This function is optional.

**Table 5** algo
Description	Quantization algorithm
Type	object
Value	-
Command-Line Options	ulq_quantize: ULQ clipping-based quantization algorithm arq_retrain: ARQ quantization algorithm
Recommended Value	Set to ulq_quantize for activation quantization or arq_retrain for weight quantization.
Required/Optional	This function is optional.

**Table 6** channel_wise
Description	Whether to use different quantization factors for each channel
Specification	bool
Value	true or false
Command-Line Options	true: Channels are separately quantized using different quantization factors. false: All channels are quantized altogether using the same quantization factors.
Recommended	true
Required/Optional	This function is optional.

**Table 7** fixed_min
Description	Fixed lower bound switch for the activation quantization algorithm
Type	bool
Value	true or false
Command-Line Options	true: fixes the lower bound of the activation quantization algorithm at 0. false: does not fix the lower bound of the activation quantization algorithm. If this parameter is not included, AMCT automatically sets the lower bound of the activation quantization algorithm according to the graph structure. If this parameter is included, set this parameter for each layer to be quantized as follows: true if the upstream layer is ReLU; false otherwise.
Recommended Value	Do not include this parameter.
Required/Optional	This function is optional.

**Table 8** clip_max
Description	Upper bound for the activation quantization algorithm
Type	float
Value	clip_max>0 Controls the upper bound max based on the distribution of the activation values at different layers. The recommended value range is as follows: 0.3max~1.7max
Command-Line Options	If this parameter is included, the upper bound of the clipping-based activation quantization algorithm is fixed. If this parameter is not included, the upper bound is learned using the IFMR algorithm.
Recommended Value	Do not include this parameter.
Required/Optional	This function is optional.

**Table 9** clip_min
Description	Lower bound for the activation quantization algorithm
Specification	float
Value	clip_min<0 Controls the lower bound min based on the distribution of the activation values at different layers. The recommended value range is as follows: 0.3min~1.7min
Command-Line Options	If this parameter is included, the lower bound of the clipping-based activation quantization algorithm is fixed. If this parameter is not included, the lower bound is learned using the IFMR algorithm.
Recommended Value	Do not include this parameter.
Required/Optional	Optional

Parent topic: Performs quantization aware training.