save_model

Function Usage

Inserts operators such as AscendQuant and AscendDequant based on the quantization factor record file record_file and the source PB model to be quantized, and saves the model as a PB model file that can be used for both accuracy simulation in the TensorFlow environment and offline inference.

Constraints

This API can be called only after batch_num forward passes are completed. Failure to do so may lead to incorrect quantization factors and thus unsatisfactory quantization result.
This API receives only the model file in .pb format. You need to convert the model to be quantized to the .pb format in advance.
This API requires the input of a quantization factor record file, which is generated in the quantize_model phase and has its factor values filled in the model inference phase.

Prototype

save_model(pb_model, outputs, record_file, save_path)

Parameters

Option	Input/Return	Description	Restriction
pb_model	Input	Source .pb model file to be quantized.	A string
outputs	Input	List of output operators of the graph.	A list of strings.
record_file	Input	Directory of the quantization factor record file, including the file name. Generate a quantized model file based on the file, quantization configuration file, and source .pb model file.	A string
save_path	Input	Model save path. Must include the prefix of the model name, for example, *./quantized_model/model**.	A string.

Returns

None

Outputs

A .pb model file that serves for both accuracy simulation in the TensorFlow environment and offline inference on Ascend AI Processor.

When quantization is performed again, the preceding files output by the API will be overwritten.

Examples

import amct_tensorflow as amct
# Perform network inference and complete quantization during the inference.
for i in batch_num:
    sess.run(outputs, feed_dict={inputs: inputs_data})
# Insert the API call and save the quantized model as an ONNX file.
amct.save_model(pb_model="./user_model.pb",
                outputs=["model/outputs"],
                record_file="./record_scale_offset.txt",
                save_path="./inference/model")

Parent topic: PTQ APIs