Simplified Distillation Configuration File

Table 1 describes the fields in the distill_config_pytorch.proto file. Find the file in /amct_pytorch/proto/distill_config_pytorch.proto under the AMCT installation directory.

**Table 1** distill_config_pytorch.proto
Parameter	Required/Optional	Type	Field	Description
AMCTDistillConfig	-	-	-	Simplified configuration for AMCT distillation.
	Optional	UInt32	batch_num	Number of distillation batches, which is used by IFMR to accumulate data and calculate quantization factors.
	Optional	UInt32	group_size	Minimum number of distillation units in a distillation block.
	Optional	Boolean	data_dump	Whether to enable block input and output dump for the teacher network.
	Repeated	DistillGroup	distill_group	User-defined distillation structure.
	Optional	DistillDataQuantConfig	distill_data_quant_config	Activation quantization configuration for distillation.
	Optional	DistillWeightQuantConfig	distill_weight_quant_config	Weight quantization configuration for distillation.
	Repeated	DistillOverrideLayer	distill_override_layers	Layers to override.
	Repeated	DistillOverrideLayerType	distill_override_layer_types	Type of the layer to override.
	Repeated	String	quant_skip_layers	Distillation of layers that do not need to be quantized.
	Repeated	String	quant_skip_layer_types	Distillation of operator types that do not need to be quantized.
DistillGroup	-	-	-	User-defined distillation structure. The distillation structure supports only operators of the torch.nn.Module type.
	Required	String	start_layer_name	Start layer of the user-defined distillation structure.
	Required	String	end_layer_name	End layer of the user-defined distillation structure.
DistillDataQuantConfig	-	-	-	Activation quantization configuration for distillation.
DistillDataQuantConfig	-	ActULQquantize	ulq_quantize	Activation quantization algorithm. Currently, only ULQ is supported.
ActULQquantize	-	-	-	ULQ algorithm for activation quantization. For details about the algorithm, see ULQ Algorithm.
	Optional	ClipMaxMin	clip_max_min	Initial upper and lower bounds. IFMR is used for initialization by default.
	Optional	Boolean	fixed_min	Whether to fix the lower bound at 0. Set to true for ReLU or false for other algorithms.
	Optional	DataType	dst_type	Type of the quantized data, which can be INT8 (default) or INT4. The current version supports only INT8 quantization.
ClipMaxMin	-	-	-	Initial upper and lower bounds.
	Required	Float	clip_max	Initial upper bound.
	Required	Float	clip_min	Initial lower bound.
DistillWeightQuantConfig	-	-	-	Weight quantization configuration for distillation.
	-	ARQDistill	arq_distill	ARQ algorithm.
	-	WtsULQDistill	ulq_distill	ULQ algorithm for weight quantization.
ARQDistill	-	-	-	ARQ algorithm configuration. For details about the algorithm, see ARQ.
	Optional	DataType	dst_type	Type of the quantized data, which can be INT8 (default) or INT4. The current version supports only INT8 quantization.
	Optional	Boolean	channel_wise	Channel-wise ARQ enable.
WtsULQDistill	-	-	-	ULQ algorithm for weight quantization. For details about the algorithm, see ULQ Algorithm.
	Optional	DataType	dst_type	Type of the quantized data, which can be INT8 (default) or INT4. The current version supports only INT8 quantization.
	Optional	Boolean	channel_wise	Channel-wise ULQ enable.
DistillOverrideLayer	-	-	-	Layer overriding configuration.
	Required	String	layer_name	Layer name.
	Optional	DistillDataQuantConfig	distill_data_quant_config	Activation quantization configuration to override.
	Optional	DistillWeightQuantConfig	distill_weight_quant_config	Weight quantization configuration to override.
DistillOverrideLayerType	-	-	-	Types of layers to override.
	Required	String	layer_type	Layer type.
	Optional	DistillDataQuantConfig	distill_data_quant_config	Activation quantization configuration to override.
	Optional	DistillWeightQuantConfig	distill_weight_quant_config	Weight quantization configuration to override.

The following is an example of the simplified configuration file (quant.cfg) for distillation:

batch_num: 1
group_size: 1
data_dump: true

distill_group: {
    start_layer_name: "layer1"
    end_layer_name: "layer2"
}

distill_data_quant_config: {
    ulq_quantize: {
        clip_max_min: {
            clip_max: 6.0
            clip_min: -6.0
        }
        fixed_min: true
        dst_type: INT8
    }
}

distill_weight_quant_config: {
    arq_distill: {
       channel_wise: true
       dst_type: INT8
    }
  }

quant_skip_layers: "layer3"
quant_skip_layer_types: "type1"

distill_override_layers : {
    layer_name: "layer4"
    distill_data_quant_config: {
        ulq_quantize: {
            clip_max_min: {
                clip_max: 3.0
                clip_min: -3.0
            }
            fixed_min: true
            dst_type: INT8
        }
    }
    distill_weight_quant_config: {
        arq_distill: {
           channel_wise: false
           dst_type: INT8     
        }
    }
}

distill_override_layer_types : {
    layer_type: "type2"
    distill_data_quant_config: {
        ulq_quantize: {
            clip_max_min: {
                clip_max: 3.0
                clip_min: -3.0
            }
            fixed_min: true
            dst_type: INT8
        }
    }
    distill_weight_quant_config: {
        ulq_distill: {
           channel_wise: false
           dst_type: INT8     
        }
    }
}

Parent topic: Reference