Model Adaptation Using convert_model API

If you have the quantization factor calculated by yourself and the original TensorFlow model, the model cannot be converted into an offline model adapted to the AI processor by using ATC. You need to use the function provided in this section to convert the model into the CANN quantization model format before using ATC to convert the CANN quantization model into an offline model adapted to the AI processor.

Adaptation Principles

The following figure shows the adaptation principles. The user implements the operations in blue, while those in gray are implemented by using convert_model APIs in AMCT. Specifically, import the package to the source TensorFlow network inference code and call APIs where appropriate for adaptation. For the adaptation example, see Sample List.

Figure 1 Model adaptation principles

Example

This example demonstrates how to use the convert_model API to adapt an original model.

Take the following steps to get started. Update the sample code based on your situation.
To reuse the following code for quantizing a different model, prepare the original model and build quantization factor record file based on user-defined quantization factors yourself.

Import the AMCT package and set the log level.

import amct_tensorflow as amct
amct.set_logging_level(print_level='info', save_level='info')

(Optional) Run inference on the original model for quantization in the TensorFlow environment based on the test dataset to validate the inference script and environment setup. (Update the sample code based on your situation.)
This step is recommended as it guarantees a properly functioning original model for inference with acceptable accuracy. You can use a subset from the test dataset to improve the efficiency.
1
user_do_inference(ori_model, test_data)

Call the convert_model API in AMCT to perform model adaptation.

This API parses the .pb model into a graph, preprocesses the graph, parses the input quantization factor record file, inserts operators such as AscendQuant and AscendDequant into the modified graph based on the quantization factors, and then saves the quantized model.

quant_model_path = './results/user_model'
record_file = './results/record.txt'
amct.convert_model(pb_model=ori_model,
		   outputs=ori_model_outputs,
       		   record_file=record_file,
		   save_path=quant_model_path)

(Optional) Run inference on the quantized model user_model_quantized.pb in the TensorFlow environment based on the test dataset to test the accuracy. (Update the sample code based on your situation.)
Compare the accuracy of the fake-quantized model with that of the original model (see 2).
1 2
quant_model = './results/user_model_quantized.pb' user_do_inference(quant_model, test_data)

Parent topic: Model Adaptation