310B部署deepseek模型推理
收藏回复举报
310B部署deepseek模型推理
t('forum.solved') 已解决
发表于2025-02-17 15:08:16
0 查看

参考该样例Ascend/ModelZoo-PyTorch - Gitee.com,在310B上部署deepseek模型,运行时出现以下错误

Traceback (most recent call last): 

  File "/usr/local/Python3.10.0/lib/python3.10/runpy.py", line 196, in _run_module_as_main 

    return _run_code(code, main_globals, None, 

  File "/usr/local/Python3.10.0/lib/python3.10/runpy.py", line 86, in _run_code 

    exec(code, run_globals) 

  File "/opt/MindIE-LLM/examples/run_fa_edge.py", line 280, in <module> 

    fa_runner = FARunner(**input_dict) 

  File "/opt/MindIE-LLM/examples/run_fa_edge.py", line 112, in __init__ 

    self.model = ModelRunner( 

  File "/opt/MindIE-LLM/atb_llm/runner/model_runner.py", line 85, in __init__ 

    load_atb_speed() 

  File "/opt/MindIE-LLM/atb_llm/utils/initial.py", line 53, in load_atb_speed 

    torch.classes.load_library(lib_path) 

  File "/usr/local/Python3.10.0/lib/python3.10/site-packages/torch/_classes.py", line 51, in load_library 

    torch.ops.load_library(path) 

  File "/usr/local/Python3.10.0/lib/python3.10/site-packages/torch/_ops.py", line 852, in load_library 

    ctypes.CDLL(path) 

  File "/usr/local/Python3.10.0/lib/python3.10/ctypes/__init__.py", line 374, in __init__ 

    self._handle = _dlopen(self._name, mode) 

OSError: /opt/MindIE-LLM/lib/libatb_speed_operations.so: undefined symbol: _ZN3atb15CreateOperationINS_5infer20CohereLayerNormParamEEEiRKT_PPNS_9OperationE 

[ERROR] 1970-01-01-00:38:20 (PID:30095, Device:-1, RankID:-1) ERR99999 UNKNOWN application exceptio

CANN版本:Ascend-cann-toolkit_8.0.0_linux-aarch64.run

kernels版本:Ascend-cann-kernels-310b_8.0.0_linux-aarch64.run

nnal版本:Ascend-cann-nnal_8.0.0_linux-aarch64.run

python版本:python3.10.0

pytorch版本:2.1.0

我要发帖子