量化方式：https://gitee.com/ascend/msit/blob/br_noncom_MindStudio_8.0.0_POC_20251231/msmodelslim/example/Qwen/README.md

python3 quant_qwen.py --model_path /mnt/nvme/DeepSeek-R1-Distill-Qwen-1.5B --save_directory /mnt/nvme/DeepSeek-R1-Distill-Qwen-1.5B-w8a8-1 --calib_file ../common/boolq.jsonl --w_bit 8 --a_bit 8

量化后生成了权重文件，使用此权重文件进行测试：

cd /usr/local/Ascend/atb-models/tests/modeltest

bash run.sh pa_fp16 performance [[512,1024]] 16 qwen /mnt/nvme/DeepSeek-R1-Distill-Qwen-1.5B-w8a8-1 1

报错：

[2025-03-05 17:48:10,212] [1770274] [281473072060992] [llm] [INFO] [flash_causal_qwen2.py-137] : >>>> qwen_QwenDecoderModel is called. 

[2025-03-05 17:48:10,848] [1770274] [281473072060992] [llm] [ERROR] [logging.py-50] : Error caught during inference: weight model.layers.0.self_attn.q_proj.bias does not exist 

[2025-03-05 17:48:10,848] [1770274] [281473072060992] [llm] [ERROR] [logging.py-50] : Stack trace: 

Traceback (most recent call last): 

  File "/usr/local/Ascend/atb-models/tests/modeltest/base/model_test.py", line 691, in run_performance_test 

    self.__get_model_or_runner(seq_len_in_pa, seq_len_out_pa, warmup_bs=self.warmup_bs) 

  File "/usr/local/Ascend/atb-models/tests/modeltest/base/model_test.py", line 3136, in __get_model_or_runner 

    self.pa_runner = PARunner(**input_dict) 

                     ^^^^^^^^^^^^^^^^^^^^^^ 

  File "/usr/local/Ascend/atb-models/examples/run_pa.py", line 97, in __init__ 

    self.model.load_weights(**kw_args) 

  File "/usr/local/Ascend/atb-models/atb_llm/runner/model_runner.py", line 161, in load_weights 

    self.model = self.model_cls(self.config, 

                 ^^^^^^^^^^^^^^^^^^^^^^^^^^^ 

  File "/usr/local/Ascend/atb-models/atb_llm/models/qwen2/flash_causal_qwen2.py", line 42, in __init__ 

    self.transformer = FlashQwenModel(config, weights, model_prefix=model_prefix, lmhead_prefix=lmhead_prefix) 

                       ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ 

  File "/usr/local/Ascend/atb-models/atb_llm/models/qwen2/modeling_qwen2.py", line 407, in __init__ 

    [ 

  File "/usr/local/Ascend/atb-models/atb_llm/models/qwen2/modeling_qwen2.py", line 408, in <listcomp> 

    FlashQwenLayer( 

  File "/usr/local/Ascend/atb-models/atb_llm/models/qwen2/modeling_qwen2.py", line 278, in __init__ 

    self.attn = FlashQwenAttention( 

                ^^^^^^^^^^^^^^^^^^^ 

  File "/usr/local/Ascend/atb-models/atb_llm/models/qwen2/modeling_qwen2.py", line 149, in __init__ 

    self.c_attn = load_column_multi( 

                  ^^^^^^^^^^^^^^^^^^ 

  File "/usr/local/Ascend/atb-models/atb_llm/utils/layers/__init__.py", line 50, in load_column_multi 

    b = [weights.get_sharded(f"{p}.bias", dim=0, gqa_size=head_size) for p in prefixes] 

        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ 

  File "/usr/local/Ascend/atb-models/atb_llm/utils/layers/__init__.py", line 50, in <listcomp> 

    b = [weights.get_sharded(f"{p}.bias", dim=0, gqa_size=head_size) for p in prefixes] 

         ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ 

  File "/usr/local/Ascend/atb-models/atb_llm/utils/weights.py", line 331, in get_sharded 

    slice_ = self._get_slice(tensor_name) 

             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ 

  File "/usr/local/Ascend/atb-models/atb_llm/utils/weights.py", line 718, in _get_slice 

    filename, tensor_name = self.get_filename(tensor_name) 

                            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ 

  File "/usr/local/Ascend/atb-models/atb_llm/utils/weights.py", line 146, in get_filename 

    raise AssertionError(f"weight {tensor_name} does not exist") 

AssertionError: weight model.layers.0.self_attn.q_proj.bias does not exist 

[2025-03-05 17:48:10,848] [1770274] [281473072060992] [llm] [INFO] [model_test.py-739] : Total passed cases: 0 

[2025-03-05 17:48:10,849] [1770274] [281473072060992] [llm] [INFO] [model_test.py-740] : Failed case details: batch_size: 16, seq_len_in: 512, seq_len_out: 1024 

使用量化前的权重可以跑成功

求助大佬应该怎么量化呢？？？