perf_based_auto_calibration

Applicable Products

Product	Supported
Ascend 950PR/Ascend 950DT	x
Atlas A3 training products/Atlas A3 inference products	x
Atlas A2 training products/Atlas A2 inference products	x
Atlas 200I/500 A2 inference products	x
Atlas inference products	x
Atlas training products	x

Description

Tests the original model and quantized model on the board, searches for a mixed-precision quantization configuration that yields optimal performance, and outputs the mixed-precision quantized model.

Prototype

perf_based_auto_calibration(original_model_file, quantize_model_file, sampler_config_file, save_dir, strategy='BatchRollBack')

Parameters

Parameter	Input/Output	Description
original_model_file	Input	Definition file (.onnx) of the dequantized ONNX model. A string.
quantize_model_file	Input	Definition file (.onnx) of the dequantized ONNX model. A string.
sampler_config_file	Input	Performance sampler configuration file provided by users. For details about this file and the configuration example, see Performance Sampler Configuration File. A string.
save_dir	Input	Path for saving the model after performance unquantization. A string.
strategy	Input	Policy for searching for quantization configurations that meet performance requirements. A string or a Python instance (PerfStrategyBase). Default: BatchRollBack

Returns

None

Example

import amct_onnx as amct
def main():
    args_check(args)
    model_file = args.model_file_name
    quant_model_file = args.quant_model_file_name
    save_dir = args.path_dir
    sampler_config_file = args.sampler_config_file
 
    amct.perf_based_auto_calibration(
        model_file, quant_model_file, sampler_config_file, save_dir)

Flush files:

A fake-quantized model file for accuracy simulation on ONNX Runtime with the file name containing the fake_quant keyword.
A deployable model file with the file name containing the deploy keyword. The model can be deployed on the AI processor after being converted by ATC.

Parent topic: PTQ APIs