一、问题现象(附报错日志上下文):
300Iduo卡R1蒸馏版Qwen7B稀疏量化报错AttributeError: 'Qwen2Attention' object has no attribute 'fa_quantizer'
日志:
[root@hostname-iyznn Qwen]# python3 quant_qwen.py --model_path /home/weights/R17B/ --save_directory /home/weights/R17BW8A8S --calib_file ../common/boolq.jsonl --w_bit 4 --a_bit 8 --fraction 0.011 --co_sparse True --device_type npu --use_sigma True --is_lowbit True
2025-04-17 10:09:28,904 - msmodelslim-logger - WARNING - The current CANN version does not support importing the migration and migration_vit packages.
2025-04-17 10:09:29,093 - msmodelslim-logger - WARNING - The current CANN version does not support recall_window method.
2025-04-17 10:09:29,102 - msmodelslim-logger - WARNING - The current CANN version does not support LayerSelector quantile method.
Loading checkpoint shards: 100%|█████████████████████████████████████████████████████████████████| 2/2 [00:03<00:00, 1.66s/it]
Traceback (most recent call last):
File "/home/code/msit/msmodelslim/example/Qwen/quant_qwen.py", line 311, in
quantifier.convert(tokenized_calib_data, save_directory, args.disable_level, part_file_size=args.part_file_size,
File "/home/code/msit/msmodelslim/example/Qwen/quant_qwen.py", line 239, in convert
calibrator = Calibrator(self.model, self.quant_config, calib_data=tokenized_data, disable_level=disable_level)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib64/python3.11/site-packages/torch/utils/_contextlib.py", line 115, in decorate_context
return func(*args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.11/site-packages/msmodelslim/pytorch/llm_ptq/llm_ptq_tools/quant_tools.py", line 160, in init
self.rollback_names_process(model)
File "/usr/local/lib/python3.11/site-packages/msmodelslim/pytorch/llm_ptq/llm_ptq_tools/quant_tools.py", line 414, in rollback_names_process
elif is_attn_module_and_then_check_quantizer(module, name):
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.11/site-packages/msmodelslim/pytorch/llm_ptq/llm_ptq_tools/fa_quant.py", line 315, in is_attn_module_and_then_check_quantizer
if not (hasattr(module, "fa_quantizer") or isinstance(module.fa_quantizer, FAQuantizer)):
^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib64/python3.11/site-packages/torch/nn/modules/module.py", line 1695, in getattr
raise AttributeError(f"'{type(self).name}' object has no attribute '{name}'")
AttributeError: 'Qwen2Attention' object has no attribute 'fa_quantizer'
[ERROR] 2025-04-17-10:09:44 (PID:95191, Device:0, RankID:-1) ERR99999 UNKNOWN application exception
二、软件版本:
npu-smi 24.1.0.1 Version: 24.1.0.1
CANN:
package_name=Ascend-cann-toolkit
version=8.0.0
innerversion=V100R001C20SPC001B251
compatible_version=[V100R001C15],[V100R001C17],[V100R001C18],[V100R001C19],[V100R001C20]
arch=aarch64
os=linux
path=/usr/local/Ascend/ascend-toolkit/8.0.0/aarch64-linux
OS:openEuler release 22.03 (LTS-SP3)
一、问题现象(附报错日志上下文):
300Iduo卡R1蒸馏版Qwen7B稀疏量化报错AttributeError: 'Qwen2Attention' object has no attribute 'fa_quantizer'
日志:
[root@hostname-iyznn Qwen]# python3 quant_qwen.py --model_path /home/weights/R17B/ --save_directory /home/weights/R17BW8A8S --calib_file ../common/boolq.jsonl --w_bit 4 --a_bit 8 --fraction 0.011 --co_sparse True --device_type npu --use_sigma True --is_lowbit True
2025-04-17 10:09:28,904 - msmodelslim-logger - WARNING - The current CANN version does not support importing the migration and migration_vit packages.
2025-04-17 10:09:29,093 - msmodelslim-logger - WARNING - The current CANN version does not support recall_window method.
2025-04-17 10:09:29,102 - msmodelslim-logger - WARNING - The current CANN version does not support LayerSelector quantile method.
Loading checkpoint shards: 100%|█████████████████████████████████████████████████████████████████| 2/2 [00:03<00:00, 1.66s/it]
Traceback (most recent call last):
File "/home/code/msit/msmodelslim/example/Qwen/quant_qwen.py", line 311, in
quantifier.convert(tokenized_calib_data, save_directory, args.disable_level, part_file_size=args.part_file_size,
File "/home/code/msit/msmodelslim/example/Qwen/quant_qwen.py", line 239, in convert
calibrator = Calibrator(self.model, self.quant_config, calib_data=tokenized_data, disable_level=disable_level)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib64/python3.11/site-packages/torch/utils/_contextlib.py", line 115, in decorate_context
return func(*args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.11/site-packages/msmodelslim/pytorch/llm_ptq/llm_ptq_tools/quant_tools.py", line 160, in init
self.rollback_names_process(model)
File "/usr/local/lib/python3.11/site-packages/msmodelslim/pytorch/llm_ptq/llm_ptq_tools/quant_tools.py", line 414, in rollback_names_process
elif is_attn_module_and_then_check_quantizer(module, name):
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.11/site-packages/msmodelslim/pytorch/llm_ptq/llm_ptq_tools/fa_quant.py", line 315, in is_attn_module_and_then_check_quantizer
if not (hasattr(module, "fa_quantizer") or isinstance(module.fa_quantizer, FAQuantizer)):
^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib64/python3.11/site-packages/torch/nn/modules/module.py", line 1695, in getattr
raise AttributeError(f"'{type(self).name}' object has no attribute '{name}'")
AttributeError: 'Qwen2Attention' object has no attribute 'fa_quantizer'
[ERROR] 2025-04-17-10:09:44 (PID:95191, Device:0, RankID:-1) ERR99999 UNKNOWN application exception
二、软件版本:
npu-smi 24.1.0.1 Version: 24.1.0.1
CANN:
package_name=Ascend-cann-toolkit
version=8.0.0
innerversion=V100R001C20SPC001B251
compatible_version=[V100R001C15],[V100R001C17],[V100R001C18],[V100R001C19],[V100R001C20]
arch=aarch64
os=linux
path=/usr/local/Ascend/ascend-toolkit/8.0.0/aarch64-linux
OS:openEuler release 22.03 (LTS-SP3)