Documentation

Why Is the Message "RuntimeError: record_file is empty, no layers to be quantized" During Model Saving?

Symptom

In the PTQ scenario, the shift_bit field of a layer cannot be found in the quantization factor record file. As a result, the subsequent quantization model cannot be saved, and the quantization process is terminated. The message is as follows:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
Traceback (most recent call last):
  File "./GenAIModel/amt/tools/tmp_amct_tools.py", line 457, in wrapper
    func(*args, **kwargs)
  File "/raid/AMCT/demo/new_project/test/test_tf_iter37_d16w8.py", line 1806, in test_xxx
    run_case(config, acc=True)
  File "/raid/AMCT/demo/new_project/test/test_tf_iter37_d16w8.py", line 130, in run_case
    res = tf_api_compress(model, input_data, quant_cfg=quant_cfg,data_config=data_config, nuq=nuq, per_channel=per_channel)
  File "./GenAIModel/amt/model_process/infer_quant.py", line 754, in tf_api_compress
    save_model(model, out_puts, record_file, f'{save_path}/{case_name}' )
  File "/raid/AMCT/daily/daily_env/lib/python3.9/site-packages/amct_tensorflow/common/utils/check_params.py", line 43, in wrapper
    return func(*args, **kwargs)
  File "/raid/AMCT/daily/daily_env/lib/python3.9/site-packages/amct_tensorflow/interface/save_model.py", line 68, in save_model
    save_model_inner(pb_model, outputs, record_file, save_path)
  File "/raid/AMCT/daily/daily_env/lib/python3.9/site-packages/amct_tensorflow/interface/save_model.py", line 102, in save_model_inner
    raise RuntimeError(
RuntimeError: record_file is empty, no layers to be quantized. please ensure calibration is finished by checking information!

Possible Cause

If the tail-layer node of the network is a quantization node and the inference output node is not modified during quantization inference, this problem occurs. However, when quantize_model is called, a warning message will display if the output node has been changed.

Solution

When quantize_model is called, a warning message will display upon output node changes. During the calibration, modify the output node as prompted.

Parent topic: FAQ