Documentation

Operator Accuracy Pre-Check

Once a potential accuracy issue with an operator is identified, you can use the msit llm opcheck tool to perform a pre-check on the operator's accuracy, determining whether the accuracy of the ATB operator meets the required standards. For details about how to use the msit llm opcheck tool and the parameters, see Accuracy Precheck Usage.

  1. Pre-check the accuracy of the LinearOperation operator.

    Example:

    msit llm opcheck -i {OUTPUT_DIR}/msit_dump_{TIMESTAMP}/tensors/{device_id}_{PID}/{TID}/ -ids 50_2_4_0 -opname LinearOperation  -o "Path for saving the pre-check result data"

    ids indicates the specified pre-check tensor index, which is the prefix of its parent directory. If the tensor path is as follows:

    {OUTPUT_DIR}/msit_dump_{TIMESTAMP}/tensors/{device_id}_{PID}/3/50_Decoder_layer/2_MlpGateUpWeightPack/4_LinearNoQuant/0_LinearOperation/after/outtensor0.bin, the ids is 50_2_4_0.

  2. The check result is printed on the screen, as shown in Figure 1.
    Figure 1 Check result

    Save the pre-check result to the path specified by 1. Open the pre-check result file. The data is shown in Figure 2.

    Figure 2 Pre-check result
  3. Check whether the operator precision meets the requirements. For details, see the Precision Standards.
    • If the operator's precision meets the standards, it indicates that the error is caused by variations in the inputs between the benchmark and the NPU operator. To resolve this, you can modify the data types used during the model's inference computation to improve the model's data precision.
    • If the operator precision does not meet the requirements:
      1. Check whether the operator's output tensor has very small values (for example, in SelfAttentionOperation and PagedAttentionOperation). Such operators often result in large relative error values, which can cause precision comparison results to fail. In such cases, it is recommended that you use the -metric option to specify other metrics for a comprehensive evaluation of the operator's precision.
      2. Check and align the operator input parameters. If the parameters do not meet the requirements, ask experienced operator developers to analyze the issue. You can try to resolve the problem by referring to Fault Locating Process in CANN AscendCL Application Software Development Guide (C&C++).