Accuracy Pre-Check

Prerequisites

• You have performed operations in Environment Setup.
• You have performed operations in Accuracy Data Collection and obtained the accuracy data of the Ascend NPU environment in the PyTorch training scenario.

Performing Pre-Check

1. Prepare data.

   Copy the accuracy data dumped from the Ascend NPU environment to the GPU environment. (This ensures that the accuracy data used for pre-check is consistent.)

2. Start the pre-check.
   Run the run_ut command in the GPU and Ascend NPU environments to perform the pre-check. (In the pre-check scenario, the GPU environment needs to use the accuracy data copied from the Ascend NPU environment.)

   msprobe -f pytorch run_ut -api_info ./dump_data/step0/rank/dump.json -o ./checker_result

   If the following log is displayed, the pre-check is complete:

   Successfully completed run_ut/multi_run_ut

   Two CSV files are generated in the path specified by the -o parameter: accuracy_checking_details_{timestamp}.csv and accuracy_checking_result_{timestamp}.csv.

   The two files are the intermediate results of the pre-check. You need to perform the next step to obtain the final results of the pre-check.

3. Compare the pre-check results.

   Compare the pre-check results of the NPU and GPU to check whether the NPU data contains information about APIs with accuracy issues.

   Run the following command to upload the accuracy_checking_details_{timestamp}.csv file on the GPU to the NPU environment:

   msprobe -f pytorch api_precision_compare -npu ./npu/accuracy_checking_details_{timestamp}.csv -gpu ./gpu/accuracy_checking_details_{timestamp}.csv -o ./compare_result/accuracy_checking

4. Analyze the pre-check result.

   api_precision_compare generates two .csv files in the ./compare_result/accuracy_checking directory.

   • The api_precision_compare_result_{timestamp}.csv file describes the API performance in various comparison algorithms. The following gives an example.
     Figure 1 api_precision_compare_result_{timestamp}
     
   • The api_precision_compare_details_{timestamp}.csv file indicates whether each API has passed the test. The following gives an example.
     Figure 2 api_precision_compare_details_1
     
     Figure 3 api_precision_compare_details_2
     

     For details about field meanings in the comparison results, see Pre-Check Result Comparison.