Performing Accuracy Comparison Between .npy Files

Introduction

Model Accuracy Analyzer supports accuracy comparison between two different .npy files. The basic requirements are as follows:

For dump data files, perform the operations described in atlasaccuracy_16_0054.html#EN-US_TOPIC_0000002502563684__en-us_topic_0244646216_section19803552124011 first.
Ensure that the shapes in the two files to be compared are the same.
Only CosineSimilarity (cosine similarity), MaxAbsoluteError (maximum absolute error), AccumulatedRelativeError (accumulated relative error), RelativeEuclideanDistance (Euclidean relative distance), MeanAbsoluteError (mean absolute error), RootMeanSquareError (root mean square error), MaxRelativeError (maximum relative error), and MeanRelativeError (mean relative error) comparison algorithms are supported.

Command Syntax

python3 msaccucmp.py file_compare -m my_dump_path -g golden_dump_path -out output

Table 1 describes the command-line options.

This function is implemented through the msaccucmp.py script, which is stored in ${INSTALL_DIR}/tools/operator_cmp/compare. Replace ${INSTALL_DIR} with the CANN component directory. For example, if the installation is performed by the root user, the default file storage path is /usr/local/Ascend/cann..

**Table 1** Command-line options
Option	Description	Mandatory (Yes/No)
-m --my_dump_path	.npy files to be compared.	Yes
-g --golden_dump_path	Benchmark .npy file to be compared.	Yes
-out --output	Directory for storing the comparison result. The format of the result file name is file_result_{timestamp}.txt. You are not advised to configure directories that are different from those of the current user to avoid privilege escalation risks.	Yes

Procedure

Log in to the CANN tool installation environment.
Go to the ${INSTALL_DIR}/tools/operator_cmp/compare directory. Replace ${INSTALL_DIR} with the CANN component directory. For example, if the installation is performed by the root user, the default file storage path is /usr/local/Ascend/cann..
Run the file_compare comparison command.
```
python3 msaccucmp.py file_compare -m my_dump_path/a.npy -g golden_dump_path/b.npy -out output
```
After the command is executed, the comparison result is displayed. See Figure 1.

Figure 1 Comparison result

You can determine whether the comparison result meets the accuracy requirements based on Computation Accuracy Evaluation Indicators.

Parent topic: Overview