Operator Accuracy Pre-Check

Once an operator with a potential accuracy issue is identified, you can use the msit llm opcheck tool to perform a pre-check on the operator's accuracy, determining whether the accuracy of the ATB operator meets the requirements. For details about how to use the msit llm opcheck tool and the optional parameters, see opcheck Single-Operator Accuracy Pre-Check User Guide.

1. Pre-check the accuracy of the LinearOperation operator.

   The following is an example. CHECK_PATH indicates the path for saving the pre-check result.

   msit llm opcheck -i {GOLDEN_DUMP_ALL_PATH}/msit_dump_{TIMESTAMP}/tensors/{device_id}_{PID}/{TID}/ -ids 50_2_4_0 -opname LinearOperation  -o {CHECK_PATH}

   ids indicates the specified pre-check tensor index, which is the prefix of its parent directory. If the tensor path is as follows:

   {GOLDEN_DUMP_ALL_PATH}/msit_dump_{TIMESTAMP}/tensors/{device_id}_{PID}/3/50_Decoder_layer/2_MlpGateUpWeightPack/4_LinearNoQuant/0_LinearOperation/after/outtensor0.bin, ids is 50_2_4_0.

2. View the test result, as shown in Figure 1.
   Figure 1 Test result
   

   Save the pre-check result to the path specified in 1. Open the pre-check result file. The data is shown in Figure 2.

   Figure 2 Pre-check result
   
3. Check whether the operator accuracy meets the standards. For details, see Accuracy Standards.
   • If the operator's accuracy meets the standards, it indicates that the error is caused by variations in the inputs between the benchmark and the NPU operator. To resolve this, you can modify the data types used during the model's inference computation to improve the model's data accuracy.
   • If the operator accuracy does not meet the standards:
     1. Check whether the operator's output tensor has very small values (for example, in SelfAttentionOperation and PagedAttentionOperation). Such operators often result in large relative error values, which can cause accuracy comparison results to fail. In such cases, it is recommended that you use the -metric option to specify other metrics for a comprehensive evaluation of the operator's accuracy.
     2. Check and align the operator input parameters. If the parameters still do not meet the requirements, consult personnel with operator development experience to analyze and resolve the issue by referring to "Troubleshooting Workflow" in Application Development Guide (C&C++).