使用MINDIE本地机多卡推理,问题求解
收藏回复举报
使用MINDIE本地机多卡推理,问题求解
t('forum.solved') 已解决
新人帖
发表于2024-11-28 15:57:56
0 查看

cke_346.png

1、根据文档内容,使用ATB_MODEL进行大模型推理;
2、操作流程如下:
     2.1 运行命令如下:bash examples/models/qwen/run_pa.sh -m /pretrained_models/Qwen2.5-72B-Instruct 
    2.2 出现问题如下所示:
W1128 15:36:48.242000 281463394424736 torch/distributed/run.py:779] ***************************************** 

Traceback (most recent call last): 

  File "<frozen runpy>", line 198, in _run_module_as_main 

  File "<frozen runpy>", line 88, in _run_code 

  File "/package/examples/run_pa.py", line 474, in <module> 

    pa_runner = PARunner(**input_dict) 

                ^^^^^^^^^^^^^^^^^^^^^^ 

  File "/package/examples/run_pa.py", line 60, in __init__ 

    self.model = ModelRunner( 

                 ^^^^^^^^^^^^ 

  File "/package/atb_llm/runner/model_runner.py", line 49, in __init__ 

    load_atb_speed() 

  File "/package/atb_llm/utils/initial.py", line 50, in load_atb_speed 

    torch.classes.load_library(lib_path) 

  File "/envs/qwen-mindie/lib/python3.11/site-packages/torch/_classes.py", line 52, in load_library 

    torch.ops.load_library(path) 

  File "/envs/qwen-mindie/lib/python3.11/site-packages/torch/_ops.py", line 1295, in load_library 

    ctypes.CDLL(path) 

  File "/envs/qwen-mindie/lib/python3.11/ctypes/__init__.py", line 376, in __init__ 

    self._handle = _dlopen(self._name, mode) 

                   ^^^^^^^^^^^^^^^^^^^^^^^^^ 

OSError: /package/lib/libatb_speed_operations.so: undefined symbol: aclnnGroupedMatmulV4GetWorkspaceSize 

[ERROR] 2024-11-28-15:36:52 (PID:2078061, Device:-1, RankID:-1) ERR99999 UNKNOWN applicaiton exception 

显示出现未知符号链接错误,实在是找不到问题所在了,特向各位大佬求助!!!


    

我要发帖子