
1、根据文档内容,使用ATB_MODEL进行大模型推理;
2、操作流程如下:
2.1 运行命令如下:bash examples/models/qwen/run_pa.sh -m /pretrained_models/Qwen2.5-72B-Instruct
2.2 出现问题如下所示:
W1128 15:36:48.242000 281463394424736 torch/distributed/run.py:779] *****************************************
Traceback (most recent call last):
File "<frozen runpy>", line 198, in _run_module_as_main
File "<frozen runpy>", line 88, in _run_code
File "/package/examples/run_pa.py", line 474, in <module>
pa_runner = PARunner(**input_dict)
^^^^^^^^^^^^^^^^^^^^^^
File "/package/examples/run_pa.py", line 60, in __init__
self.model = ModelRunner(
^^^^^^^^^^^^
File "/package/atb_llm/runner/model_runner.py", line 49, in __init__
load_atb_speed()
File "/package/atb_llm/utils/initial.py", line 50, in load_atb_speed
torch.classes.load_library(lib_path)
File "/envs/qwen-mindie/lib/python3.11/site-packages/torch/_classes.py", line 52, in load_library
torch.ops.load_library(path)
File "/envs/qwen-mindie/lib/python3.11/site-packages/torch/_ops.py", line 1295, in load_library
ctypes.CDLL(path)
File "/envs/qwen-mindie/lib/python3.11/ctypes/__init__.py", line 376, in __init__
self._handle = _dlopen(self._name, mode)
^^^^^^^^^^^^^^^^^^^^^^^^^
OSError: /package/lib/libatb_speed_operations.so: undefined symbol: aclnnGroupedMatmulV4GetWorkspaceSize
[ERROR] 2024-11-28-15:36:52 (PID:2078061, Device:-1, RankID:-1) ERR99999 UNKNOWN applicaiton exception
显示出现未知符号链接错误,实在是找不到问题所在了,特向各位大佬求助!!!
1、根据文档内容,使用ATB_MODEL进行大模型推理;
2、操作流程如下:
2.1 运行命令如下:bash examples/models/qwen/run_pa.sh -m /pretrained_models/Qwen2.5-72B-Instruct
2.2 出现问题如下所示:
W1128 15:36:48.242000 281463394424736 torch/distributed/run.py:779] *****************************************
Traceback (most recent call last):
File "<frozen runpy>", line 198, in _run_module_as_main
File "<frozen runpy>", line 88, in _run_code
File "/package/examples/run_pa.py", line 474, in <module>
pa_runner = PARunner(**input_dict)
^^^^^^^^^^^^^^^^^^^^^^
File "/package/examples/run_pa.py", line 60, in __init__
self.model = ModelRunner(
^^^^^^^^^^^^
File "/package/atb_llm/runner/model_runner.py", line 49, in __init__
load_atb_speed()
File "/package/atb_llm/utils/initial.py", line 50, in load_atb_speed
torch.classes.load_library(lib_path)
File "/envs/qwen-mindie/lib/python3.11/site-packages/torch/_classes.py", line 52, in load_library
torch.ops.load_library(path)
File "/envs/qwen-mindie/lib/python3.11/site-packages/torch/_ops.py", line 1295, in load_library
ctypes.CDLL(path)
File "/envs/qwen-mindie/lib/python3.11/ctypes/__init__.py", line 376, in __init__
self._handle = _dlopen(self._name, mode)
^^^^^^^^^^^^^^^^^^^^^^^^^
OSError: /package/lib/libatb_speed_operations.so: undefined symbol: aclnnGroupedMatmulV4GetWorkspaceSize
[ERROR] 2024-11-28-15:36:52 (PID:2078061, Device:-1, RankID:-1) ERR99999 UNKNOWN applicaiton exception
显示出现未知符号链接错误,实在是找不到问题所在了,特向各位大佬求助!!!