Issue Description

The objective of accuracy verification in traditional model inference is to ensure that the model performs correctly on the Ascend platform. This is evaluated by comparing the inference results obtained on the Ascend platform with the benchmark results. This comparison helps determine whether accuracy issues occur after the model is migrated from a non-Ascend platform to the Ascend platform. Benchmark results refer to the model outputs generated on a non-Ascend platform before migration.

Most accuracy issues of traditional models can be attributed to operator problems or model conversion problems. However, because the model conversion process mainly involves operator conversion, fusion, and optimization, most accuracy issues can ultimately be summarized as operator problems. Operator issues can be classified into several types, such as low accuracy, data overflow, operator implementation differences, unreasonable fusion patterns, and hardware differences.

Common accuracy issues and their possible causes are as follows:

• The model output differs significantly from expectations.

  This is usually caused by issues such as operator accuracy overflow or incorrect fusion patterns.

• The model output differs slightly from the benchmark.

  This is usually caused by accumulated errors, typically resulting from operator implementations or data accuracy issues.

• The model output matches the benchmark, but differs significantly in actual applications or downstream tasks.

  The preprocessing and postprocessing at the service layer are inconsistent with the benchmark. You need to analyze these differences and align them accordingly.

• The model is normal, but the model output differs greatly from the benchmark after the hardware environment is upgraded.

  This issue is typically caused by incorrect environment configurations. The hardware environment must match the software environment such as the CANN package.

Although these issues have different symptoms, they can be located using the traditional model accuracy analysis methods described in this document.