Why Do I See "[IfmrQuantWithoutOffset] scale is illegal" During Calibration?
Symptom
When the ONNX Runtime framework is used to perform intermediate calibration model inference, the scale computed by the quantization algorithm is improper due to the invalid input data range. As a result, the calibration fails and the calibration process is terminated.
Possible Cause
The original data is not in the range 
. (EPSILON includes DBL_EPSILON double type and FLT_EPSILON float type. Currently, the FLT_EPSILON type is used.)
The AMCT quantization supports the maximum value 
obtained through calculation, because Ascend AI Processor quantization uses multiplication calculation: 
. If scale is greater than 
, then 
is less than FLT_EPSILON. In this case, the quantization result is unreliable. Therefore, the AMCT quantization algorithm supports original activation quantization only within the range 
. Otherwise, an error message is displayed, indicating that the range is not supported.
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 | [ERROR][IfmrQuantWithoutOffset][183] [IfmrQuantWithoutOffset]scale is illegal. [ERROR][ProcessScale][64] Not support scale greater than 1 / FLT_EPSILON. currentScale: 1513411575808.000000 [ERROR][IfmrQuantWithoutOffset][183] [IfmrQuantWithoutOffset]scale is illegal. [ERROR][ProcessScale][64] Not support scale greater than 1 / FLT_EPSILON. currentScale: 1492095991808.000000 [ERROR][CalScaleWithoutOffset][146] [CalScaleWithoutOffset]scale failed. [ERROR][IfmrQuantCalibration][264] CalScaleWithoutOffset failed. [ERROR][DoCalibration][126] Do IFMR calibration failed, error code: -65519. Traceback (most recent call last): File "/g/daily/xxx/bin/amct_onnx", line 8, in <module> sys.exit(main()) File "/g/daily/xxx/lib/python3.9/site-packages/amct_onnx/cmd_line_tools/command.py", line 159, in main calibration_function(args_var) File "/g/daily/xxx/lib/python3.9/site-packages/amct_onnx/cmd_line_tools/command.py", line 113, in calibration_function amct.save_model( File "/g/daily/xxx/lib/python3.9/site-packages/amct_onnx/common/utils/check_params.py", line 43, in wrapper return func(*args, **kwargs) File "/g/daily/xxx/lib/python3.9/site-packages/amct_onnx/quantize_tool.py", line 277, in save_model RecordFileParser.cmp_config(records) File "/g/daily/xxx/lib/python3.9/site-packages/amct_onnx/parser/parse_record_file.py", line 96, in cmp_config raise RuntimeError( RuntimeError: batchmatmul_7 is quantizable in config but has no record in record_file |
Solution
Skip the quantization layers listed in the log, for example, the batchmatmul_7 layer in the preceding log example.