文档

During IFMR Activation Quantization, When "inf or NaN Value" or "xxx Calculate Scale Failed" Exists, an Error Is Reported.

Symptom

When the TensorFlow framework is used to perform intermediate calibration model inference, the scale computed by the quantization algorithm is improper due to the invalid input data range. As a result, the calibration fails and the TensorFlow calibration process is terminated.

  • During IFMR activation quantization, error incurred by inf or NaN values is as follows:
     1
 2
 3
 4
 5
 6
 7
 8
 9
10
    tensorflow.python.framework.errors_impl.InvalidArgumentError: 2 root error(s) found.
  (0) Invalid argument: Error: Exit Infinite value before data quantization! : Tensor had NaN values
	 [[{{node CheckNumerics_33}}]]
	 [[detector/yolo-v3-tiny/Conv_2/act_quant/act_quant_IFMR/cond/StringFormat/Switch/_69]]
  (1) Invalid argument: Error: Exit Infinite value before data quantization! : Tensor had NaN values
	 [[{{node CheckNumerics_33}}]]
0 successful operations.
0 derived errors ignored.

During handling of the above exception, another exception occurred:
  • error incurred by "calculate scale failed" is as follows:

Possible Cause

The cause of "calculate scale failed" is as follows:

The original data is not in the range . (EPSILON includes DBL_EPSILON double type and FLT_EPSILON float type. Currently, the FLT_EPSILON type is used.)

The AMCT quantization supports the maximum value obtained through calculation, because Ascend AI Processor quantization uses multiplication calculation: . If scale is greater than , then is less than FLT_EPSILON. In this case, the quantization result is unreliable. Therefore, the AMCT quantization algorithm supports original activation quantization only within the range . Otherwise, an error message is displayed, indicating that the range is not supported.

Solution

Skip the quantization layers listed in the log, for example, the conv_1 layer in the preceding log example.

Parent topic: FAQ