Suggestions on Adding Calibration Datasets

During accuracy-based automatic quantization, you can manually change the value of batch_num in the quantization configuration, based on the batch size and the number of images required for quantization, to adjust the datasets used for calibration. batch_num controls the number of batches of data used for quantization. batch_num = total_nums/batch_size, where total_nums indicates the total number of images, and batch_size indicates the number of images used in each batch.

Generally, a larger value of batch_num indicates more data samples used for quantization and a smaller accuracy drop of the quantized model. However, excessive data does not necessarily improve accuracy, but certainly consumes more memory and reduces the quantization speed, hence resulting in insufficient memory, video RAM, and thread resources. An optimal tradeoff is achieved when the product of batch_num and batch_size is 16 or 32.