MIN-MAX Algorithm

This algorithm calculates the maximum and minimum values of the activation and weight, resizes the activation to the boundary value range of the quantization bit width based on the maximum and minimum values, and then calculates the corresponding quantization factors scale and offset using the quantization formula.

During initialization, the maximum and minimum values of the weight are calculated offline based on weight_granularity (for details, see config). The scale_w and offset_w values are obtained offline and added to the record object. In the forward pass, the maximum and minimum values of the current batch are obtained and the global maximum and minimum values are updated. After obtaining the minimum and maximum values of the weight and activation, this algorithm calculates scale and offset according to the following formula. (For the weight, only the symmetric scenario is supported, and the offset is 0.)

• Symmetric quantization

  scale=

  if abs(max) > abs(min)

  abs(max)/(2^{^}(quant_bit-1)-1)

  else

  abs(max)/(2^{^}(quant_bit-1))

  offset=0

• Asymmetric quantization

  scale=(max-min)/(2^{^}quant_bit-1)

  offset=-2^{^}(quant_bit-1)-min/scale