文档

Accuracy Comparison

Data Preparation

Prepare an original .npy data file of the Caffe ResNet-50 network and dump its data (click here for details). You can also obtain the original .npy data file by using other methods. Upload the original .npy data file to the project_directory/model directory on the MindStudio installation server as the MindStudio installation user.

Procedure

  1. On the AMIT process page, click Model Accuracy Analyse to enter the parameter configuration page for accuracy comparison.
  2. Configure comparison parameters. Figure 1 shows the detailed configurations.
    Figure 1 Page for configuring comparison parameters
    Table 1 Configuration parameters

    Parameter

    Description

    Output Path

    Path for storing the comparison result.

    Analysis Mode

    Accuracy comparison and analysis mode. In this sample, select NPU vs GPU/CPU.

    Framework

    In this sample, select Caffe.

    NPU Dump

    Directory for storing the dump data file of a model running on the NPU. For details about its directory, see Offline Model Dump.

    Model File

    In the inference scenario, select the .om file. For details about its directory, see the output path specified in 2.

    Ground Truth

    Directory for storing the .npy file in Data Preparation.

    Algorithm

    Comparison algorithm dimension. Possible values are:

    • Cosine Similarity (selected by default)
    • Relative Euclidean Distance (selected by default)
    • Absolute Error (selected by default). Corresponding comparison algorithms are as follows:
      • Max Absolute Error
      • Mean Absolute Error
      • Root Mean Square Error
    • Relative Error (selected by default). Corresponding comparison algorithms are as follows:
      • Max Relative Error
      • Mean Relative Error
      • Accumulated Relative Error
    • Kullback Leibler Divergence (not selected by default)
    • Standard Deviation (not selected by default)
  3. Click Start to start the comparison.

    If an error related to Protobuf is reported during comparison, install Protobuf of the corresponding version as prompted.

Result Display

Figure 2 shows the vector comparison result.

For details about the vector comparison result window and fields, see "Model Accuracy Analyzer > Tensor Comparison > Network-wide Comparison > Comparison Results" in the MindStudio User Guide.

Figure 2 Vector comparison result