Comparison Scenarios

Table 1 lists the vector accuracy comparison scenarios. Prepare corresponding files based on each specific scenario. For details about how to obtain the files, see the following sections about data preparation:

Generally, Model Accuracy Analyzer obtains a dump data file generated during model running on the Ascend AI Processor and compares it with an .npy data file generated during model running on the GPU/CPU. To compare two .npy data files, refer to Converting Dump Files into .npy Files to convert a dump file to an .npy file.

**Table 1** Data preparation for tensor comparison
No.	Data to Be Compared (NPU Dump)	Standard Data (Ground Truth)	Model File/Quantization Fusion File
Inference scenarios
1	Dump data of the non-quantized offline model running on the Ascend AI Processor	.npy file of the non-quantized Caffe model	Non-quantized offline model file (.om)
2	Dump data of the non-quantized offline model running on the Ascend AI Processor	.npy file of the non-quantized TensorFlow model	Non-quantized offline model file (.om)
3	Dump data of the non-quantized offline model running on the Ascend AI Processor	.npy file of the non-quantized ONNX model	Non-quantized offline model file (.om)
4	Dump data of the quantized offline model running on the Ascend AI Processor	.npy file of the non-quantized Caffe model	Quantized offline model file (.om) Quantization fusion pattern file (.json) after model compression
5	Dump data of the quantized offline model running on the Ascend AI Processor	.npy file of the quantized Caffe model	Quantized offline model file (.om)
6	Dump data of the quantized offline model running on the Ascend AI Processor	Dump data of the quantized offline model running on the Ascend AI Processor	In this scenario, you are advised to use different CANN software versions, different versions of the same model, or model data before and after optimization for comparison. Generally in this case, you do not need to specify the model file or fusion pattern file. To compare the accuracy of dump data before and after operator fusion is enabled for offline model conversion, you need to specify the operator mapping files (.json) or offline model files (*.om) with operator fusion enabled and disabled, respectively.
6	Dump data of the quantized offline model running on the Ascend AI Processor	Dump data of the quantized offline model running on the Ascend AI Processor
Training scenarios
1	Dump data of the model running on the Ascend AI Processor	.npy file of the TensorFlow network	Computational graph file (*.txt)

Parent topic: Data Preparation