Output Results and Tuning Suggestions

Float16 Overflow/Underflow Detection

Detects overflow/underflow errors in Float16 data of the comparison data. If overflow/underflow data exists, an expert suggestion is output.

Scenario restriction: Overflow/Underflow detection is not supported in comparison of .npy files of the quantized and non-quantized Caffe models.

Comparison result:

Float16 data overflow/underflow occurs in the data of the operator whose ID is 228.

Analysis result of Advisor:

Detection Type: FP16 overflow/underflow detection

Operator Index: 228

Expert Suggestion: Float16 data overflow/underflow occurs. Rectify the fault and perform comparison again.

Input Inconsistency Check

Checks the two batches of input data to be compared on the entire network are consistent. If inconsistency exists (the cosine similarity is less than 0.99), an expert suggestion is output.

Comparison result:

For the operator whose ID is 0, the input data is Input_1, but its cosine similarity is less than 0.99. In this case, the input or data preprocessing is abnormal.

Analysis result of Advisor:

Detection Type: Input inconsistency check

Operator Index: 0

Expert Suggestion: The input data of NPUDump is inconsistent with that of GroundTruth. Use the same data or check the data preprocessing process.

Network-wide Consistency Check (Faulty Node Detection)

In the network-wide comparison result, checks whether the value of a layer is less than the threshold, whether the values of subsequent layers are less than the threshold, or whether the value of the last layer is less than the threshold (the cosine similarity is less than 0.99), and outputs quantized error correction suggestions.

Comparison result:

In the data of the operator whose ID is 1174, the cosine similarity is less than 0.99, and all subsequent cosine similarities are also less than 0.99. In this case, the faulty node has accuracy problems.

Analysis result of Advisor:

Expert suggestion: The accuracy of some tensors is low, so the final result accuracy does not meet the requirement. This may be caused by quantization. You need to calibrate the data. If the problem cannot be solved, perform the following operations: After obtaining the logs, click here to contact technical support.

Network-wide Consistency Check (Single-Point Error Detection)

In the network-wide comparison result, checks that the value of a layer is less than the threshold (the cosine similarity is less than 0.99), but the final result meets the accuracy requirement, and outputs the expert suggestion.

Comparison result:

For the data whose operator ID is 195, the cosine similarity is less than 0.99, but the data at the last layer meets the accuracy requirements. In this case, a single-point error occurs.

Analysis result of Advisor:

Expert suggestion: The accuracy of some tensors is low, but the final result accuracy meets the requirement. This may be caused by internal optimization. Ignore this or perform the following operations: After obtaining the logs, click here to contact technical support..

Network-wide Consistency Check (Consistency Check)

Checks that all data in the comparison result meets the accuracy requirements, and outputs the expert suggestion.

Comparison result:

All data meets the accuracy requirements, and the model meets the accuracy requirements.

Analysis result of Advisor:

Detection Type: Network-wide consistency check

Operator Index: N/A

Expert Suggestion: All data in the comparison result meets the accuracy requirements.

If data accuracy problems occur in the use of actual models, check the postprocessing of model outputs.