Documentation

" "IfmrQuantCalibration with offset scale is illegal"" or " "IfmrQuantCalibration without offset scale is illegal"" Is Displayed During Calibration

Symptom

When the PyTorch framework is used to perform intermediate calibration model inference, the scale computed by the quantization algorithm is improper due to the invalid input data range. As a result, the calibration fails and the Caffe calibration process is terminated.

Possible Cause

  1. Data range [-inf, +inf]:

    The quantization algorithm requires forcible zero crossing. Therefore, the computed scale is inf/255=inf. In this case, the quantization factor cannot be carried. Therefore, the quantization algorithm displays an error message indicating that the data range is not supported.

    1. Scenario 1
      1
2
3
4
5
6
7
8
      I0513 16:12:37.508546 80719 ifmr_layer.cu:58] Doing layer: "Convolution1" calibration, already store 1/1 data.
I0513 16:12:37.508558 80719 ifmr_layer.cu:65] Start to do ifmr quant.
[ERROR][ProcessScale][48] Not support scale is +inf.
[ERROR][IfmrQuantCalibration][338] IfmrQuantCalibration with offset scale is illegal.
[ERROR][IfmrQuantInternel][462] IfmrQuantInternel calibration failed.
F0513 16:12:37.535028 80719 ifmr_layer.cu:70] Check failed: ret == 0 (-65519 vs. 0) Do IFMR calibration failed
*** Check failure stack trace: ***
Aborted
    2. Scenario 2
      1
2
3
4
5
6
7
8
      I0513 16:14:48.173506 81053 ifmr_layer.cu:58] Doing layer: "Convolution1" calibration, already store 1/1 data.
I0513 16:14:48.173519 81053 ifmr_layer.cu:65] Start to do ifmr quant.
[ERROR][ProcessScale][48] Not support scale is +inf.
[ERROR][IfmrQuantCalibration][359] IfmrQuantCalibration without offset scale is illegal.
[ERROR][IfmrQuantInternel][462] IfmrQuantInternel calibration failed.
F0513 16:14:48.199769 81053 ifmr_layer.cu:70] Check failed: ret == 0 (-65519 vs. 0) Do IFMR calibration failed
*** Check failure stack trace: ***
Aborted
  2. Data range: (EPSILON includes the DBL_EPSILON double type and FLT_EPSILON float type. The FLT_EPSILON type is used here.)

    The AMCT quantization supports the maximum value obtained through calculation, because Ascend AI Processor quantization uses multiplication calculation: . If scale is greater than , then is less than FLT_EPSILON. In this case, the quantization result is unreliable. Therefore, the AMCT quantization algorithm supports original activation quantization only within the range . Otherwise, an error message is displayed, indicating that the range is not supported.

     1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
    2021-05-11 09:25:42,222 - INFO - [AMCT]:[WeightsCalibrationPass]: Do layer 'Convolution1[arq_quantize]' weights calibration success!
2021-05-11 09:25:42,222 - INFO - [AMCT]:[Optimizer]: Do <class 'amct_caffe.optimizer.insert_ifmr_layer.InsertIFMRLayerPass'>
2021-05-11 09:25:42,223 - INFO - [AMCT]:[Optimizer]: Do <class 'amct_caffe.optimizer.lstm_calibration_replace.LSTMCalibrationReplacePass'>
2021-05-11 09:25:42,223 - INFO - [AMCT]:[Optimizer]: Do <class 'amct_caffe.optimizer.insert_searchn_layer.InsertSearchNLayerPass'>
2021-05-11 09:25:42,223 - INFO - [AMCT]:[Graph]: Doing whole model dump...
2021-05-11 09:25:42,225 - INFO - [AMCT]:[Utils]: The weights_file is saved in xx/data/AMCT15_CAFFE_GPU_GPU_FAQ_LEVEL1_TC_001/AMCT15_CAFFE_GPU_GPU_FAQ_LEVEL1_TC_001_tmp.caffemodel
2021-05-11 09:25:42,226 - INFO - [AMCT]:[Utils]: The model_file is saved in xx/data/AMCT15_CAFFE_GPU_GPU_FAQ_LEVEL1_TC_001/AMCT15_CAFFE_GPU_GPU_FAQ_LEVEL1_TC_001_tmp.prototxt
[ERROR][ProcessScale][52] Not support scale greater than 1 / FLT_EPSILON.
[ERROR][IfmrQuantCalibration][338] IfmrQuantCalibration with offset scale is illegal.
[ERROR][IfmrQuantInternel][462] IfmrQuantInternel calibration failed.
WARNING: Logging before InitGoogleLogging() is written to STDERR
F0511 09:25:42.292863 80432 ifmr_layer.cu:70] Check failed: ret == 0 (-65519 vs. 0) Do IFMR calibration failed
*** Check failure stack trace: ***
Aborted

Solution

Skip the quantization layers listed in the log, for example, the conv_1 layer in the preceding log example.

Parent topic: FAQ