Documentation

Comparison Scenarios

Table 1 lists the accuracy comparison scenarios supported by Tensor Comparison. Prepare corresponding files based on each specific scenario. For details about how to obtain the files, see the following sections about data preparation:

Generally, Model Accuracy Analyzer obtains a dump data file generated during model running on the Ascend AI Processor and compares it with an .npy file generated during model running on the GPU/CPU. To compare two .npy data files, refer to Converting Dump Files into .npy Files to convert a dump file to an .npy file.

Table 1 Data preparation for tensor comparison

No.

Data to Be Compared (NPU Dump)

Standard Data (Ground Truth)

Offline Model File/Quantization Fusion File

Inference scenarios

1

Dump data file of the non-quantized offline model running on the Ascend AI Processor

.npy file of the non-quantized Caffe model

Non-quantized offline model file (.om)

2

Dump data file of the non-quantized offline model running on the Ascend AI Processor

.npy file of the non-quantized TensorFlow model

Non-quantized offline model file (.om)

3

Dump data file of the non-quantized offline model running on the Ascend AI Processor

.npy file of the non-quantized ONNX model

Non-quantized offline model file (.om)

4

Dump data file of the quantized offline model running on the Ascend AI Processor

.npy file of the non-quantized Caffe model
  • Quantized offline model file (.om)
  • Quantization fusion pattern file (.json) after model compression

5

Dump data file of the quantized offline model running on the Ascend AI Processor

.npy file of the quantized Caffe model

Quantized offline model file (.om)

6

Dump data file of the quantized offline model running on the Ascend AI Processor

Dump data file of the quantized offline model running on the Ascend AI Processor
  • When comparing model data generated by different CANN software versions, different versions of the same model, or before and after optimization, you do not need to specify the model file and fusion pattern file.
  • When comparing dump data generated by model conversion before and after operator fusion is enabled, you need to specify the offline model files (.om) or operator mapping files (.json) before and after operator fusion is enabled.

Dump data file of the non-quantized offline model running on the Ascend AI Processor

Dump data file of the non-quantized offline model running on the Ascend AI Processor

Training scenarios

1

Dump data file of the training network running on the Ascend AI Processor

.npy file of the TensorFlow network

Computational graph file (*.txt)

Note: For details about quantization and non-quantization concepts, see the AMCT (Caffe).