Performing Accuracy Comparison Between .npy Files

Overview

Model Accuracy Analyzer of MindStudio supports accuracy comparison between two different .npy files. The basic requirements are as follows:

• For dump data files, perform the operations described in Converting Dump Files into .npy Files first.
• Ensure that the shapes in the two files to be compared are the same.
• Only CosineSimilarity (cosine similarity), MaxAbsoluteError (maximum absolute error), AccumulatedRelativeError (accumulated relative error), RelativeEuclideanDistance (Euclidean relative distance), MeanAbsoluteError (mean absolute error), RootMeanSquareError (root mean square error), MaxRelativeError (maximum relative error), and MeanRelativeError (mean relative error) comparison algorithms are supported.

Procedure

1. On the MindStudio menu bar, choose Ascend > Model Accuracy Analyzer > File Comparison to access the comparison parameter configuration page, as shown in the following figure. Table 1 describes the parameters.
   Figure 1 .npy file comparison configuration (Local Run)
   
   Figure 2 Run configuration (Remote Run)
   

   In Remote Run mode, after setting the run configuration, click Next to enter the comparison configuration page, as shown in Figure 3.

   Figure 3 .npy file comparison configuration (Remote Run)
   

   Table 1 Parameters

   Parameter	Description
   Run Mode	Remote Run Local Run In Windows OSs, only Remote Run is supported and this parameter is not displayed.
   Deployment	Run configuration. This parameter is available only when Remote Run is selected. You can use the Deployment function to synchronize the files and folders in a specified project to a specified directory on a remote device. For details, see Ascend Deployment.
   Remote Toolkit Path	Installation path of the Ascend-CANN-Toolkit software package in the remote operating environment, available when you select the Remote Run mode. Example: ${HOME}/Ascend/ascend-toolkit/{version}/toolkit. This parameter binds with the Deployment parameter. After you click Next, the parameter value is saved. During re-configuration, if Deployment has been configured, the parameters are automatically set and can be manually modified.
   Output Path	Local path for storing the comparison result. You can configure the path as required.
   NPU Dump(.npy)	.npy files to be compared. To compare data files remotely (Remote Run), specify the .npy file directory on the remote device.
   Ground Truth(.npy)	Benchmark .npy file to be compared.

2. Click Start. The comparison completion information is displayed in the Output area in the lower part of the MindStudio window, as shown in Figure 4.
   Figure 4 Output information
   

   The comparison result is stored in the path specified by Output Path (listed in Table 1).