Documentation

Performing Accuracy Comparison Between .npy Files

Introduction

Model Accuracy Analyzer supports accuracy comparison between two different .npy files. The basic requirements are as follows:

  • For dump data files, perform the operations described in Converting the Format of a Dump File first.
  • Ensure that the shapes in the two files to be compared are the same.
  • Only CosineSimilarity (cosine similarity), MaxAbsoluteError (maximum absolute error), AccumulatedRelativeError (accumulated relative error), RelativeEuclideanDistance (Euclidean relative distance), MeanAbsoluteError (mean absolute error), RootMeanSquareError (root mean square error), MaxRelativeError (maximum relative error), and MeanRelativeError (mean relative error) comparison algorithms are supported.

Command Syntax

python3 msaccucmp.py file_compare -m my_dump_path -g golden_dump_path -out output

Table 1 describes the command-line options.

This function is implemented through the msaccucmp.py script, which is stored in ${INSTALL_DIR}/tools/operator_cmp/compare. Replace ${INSTALL_DIR} with the actual CANN component directory. If the Ascend-CANN-Toolkit package is installed as the root user, the CANN component directory is /usr/local/Ascend/ascend-toolkit/latest..

Table 1 Command-line options

Option

Description

Mandatory (Yes/No)

-m

--my_dump_path

.npy files to be compared.

Yes

-g

--golden_dump_path

Benchmark .npy file to be compared.

Yes

-out

--output

Directory for storing the comparison result.

The format of the result file name is file_result_{timestamp}.txt.

You are not advised to configure directories that are different from those of the current user to avoid privilege escalation risks.

Yes

Procedure

  1. Log in to the CANN tool installation environment.
  2. Go to the ${INSTALL_DIR}/tools/operator_cmp/compare directory. Replace ${INSTALL_DIR} with the actual CANN component directory. If the Ascend-CANN-Toolkit package is installed as the root user, the CANN component directory is /usr/local/Ascend/ascend-toolkit/latest..
  3. Run the file_compare comparison command.
    python3 msaccucmp.py file_compare -m my_dump_path/a.npy -g golden_dump_path/b.npy -out output

    After the command is executed, the comparison result is displayed. See Figure 1.

    Figure 1 Comparison result