Why Is the Message "RuntimeError: record_file is empty, no layers to be quantized" Displayed During Model Saving?

Symptom

In the PTQ process, the shift_bit field of a layer cannot be found in the quantization factor record file. As a result, the quantized model fails to be saved and quantization stops. The message is as follows:

Traceback (most recent call last):
  File "./GenAIModel/amt/tools/tmp_amct_tools.py", line 457, in wrapper
    func(*args, **kwargs)
  File "/raid/AMCT/demo/new_project/test/test_tf_iter37_d16w8.py", line 1806, in test_xxx
    run_case(config, acc=True)
  File "/raid/AMCT/demo/new_project/test/test_tf_iter37_d16w8.py", line 130, in run_case
    res = tf_api_compress(model, input_data, quant_cfg=quant_cfg,data_config=data_config, nuq=nuq, per_channel=per_channel)
  File "./GenAIModel/amt/model_process/infer_quant.py", line 754, in tf_api_compress
    save_model(model, out_puts, record_file, f'{save_path}/{case_name}' )
  File "/raid/AMCT/daily/daily_env/lib/python3.9/site-packages/amct_tensorflow/common/utils/check_params.py", line 43, in wrapper
    return func(*args, **kwargs)
  File "/raid/AMCT/daily/daily_env/lib/python3.9/site-packages/amct_tensorflow/interface/save_model.py", line 68, in save_model
    save_model_inner(pb_model, outputs, record_file, save_path)
  File "/raid/AMCT/daily/daily_env/lib/python3.9/site-packages/amct_tensorflow/interface/save_model.py", line 102, in save_model_inner
    raise RuntimeError(
RuntimeError: record_file is empty, no layers to be quantized. please ensure calibration is finished by checking information!

Possible Cause

If the output node is quantized and no modification is made to it before inference, this issue occurs. However, when the quantize_model API is called, a warning message will display if the output node has been changed.

Solution

When the quantize_model API is called, a warning message will be displayed upon output node changes. During the calibration, modify the output node as prompted.

Parent topic: FAQs