环境:
8卡Ascend910b
npu驱动版本:23.0.3
cann版本:8.0.0
python3.10
torch版本:2.1.0
torch_npu版本:6.0.rc3
LLM:从魔塔社区下载的Qwen2.5-7B、Qwen2-7B模型
使用ATB Models对Qwen2.5-7B、Qwen2-7B模型进行推理时,输入的命令为
torchrun --nproc_per_node 8 --master_port 20030 -m examples.run_pa --model_path /data/Qwen2-7B-Instruct/ --input_texts "What's deep learning?" "Hello World." --max_batch_size 2
均报以下错误:
Traceback (most recent call last):
File "<frozen runpy>", line 198, in _run_module_as_main
File "<frozen runpy>", line 88, in _run_code
File "/usr/local/Ascend/llm_model/examples/run_pa.py", line 476, in <module>
pa_runner.warm_up()
File "/usr/local/Ascend/llm_model/examples/run_pa.py", line 204, in warm_up
self.model.forward(
File "/usr/local/Ascend/llm_model/atb_llm/runner/model_runner.py", line 157, in forward
return self.model.forward(**kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/Ascend/llm_model/atb_llm/models/base/flash_causal_lm.py", line 380, in forward
self.init_ascend_weight()
File "/usr/local/Ascend/llm_model/atb_llm/models/qwen2/flash_causal_qwen2.py", line 182, in init_ascend_weight
self.acl_encoder_operation.set_param(json.dumps({**encoder_param}))
RuntimeError: BuildModelGraph error,check ATB_LOG, ASDOPS_LOG
环境:
8卡Ascend910b
npu驱动版本:23.0.3
cann版本:8.0.0
python3.10
torch版本:2.1.0
torch_npu版本:6.0.rc3
LLM:从魔塔社区下载的Qwen2.5-7B、Qwen2-7B模型
使用ATB Models对Qwen2.5-7B、Qwen2-7B模型进行推理时,输入的命令为
torchrun --nproc_per_node 8 --master_port 20030 -m examples.run_pa --model_path /data/Qwen2-7B-Instruct/ --input_texts "What's deep learning?" "Hello World." --max_batch_size 2
均报以下错误:
Traceback (most recent call last):
File "<frozen runpy>", line 198, in _run_module_as_main
File "<frozen runpy>", line 88, in _run_code
File "/usr/local/Ascend/llm_model/examples/run_pa.py", line 476, in <module>
pa_runner.warm_up()
File "/usr/local/Ascend/llm_model/examples/run_pa.py", line 204, in warm_up
self.model.forward(
File "/usr/local/Ascend/llm_model/atb_llm/runner/model_runner.py", line 157, in forward
return self.model.forward(**kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/Ascend/llm_model/atb_llm/models/base/flash_causal_lm.py", line 380, in forward
self.init_ascend_weight()
File "/usr/local/Ascend/llm_model/atb_llm/models/qwen2/flash_causal_qwen2.py", line 182, in init_ascend_weight
self.acl_encoder_operation.set_param(json.dumps({**encoder_param}))
RuntimeError: BuildModelGraph error,check ATB_LOG, ASDOPS_LOG