本地部署和Qwen3 32B求助
收藏回复举报
本地部署和Qwen3 32B求助
t('forum.solved') 已解决
发表于2025-06-25 03:43:59
0 查看

硬件(2×Kunpeng 920 CPU + 8张Atlas 300I DUO 96GB显存 + 512GB DDR4内存,硬盘2块1.96T的固态)本地部署和Qwen3 32B,用的2.0.RC2-300I-Duo-py311-openeuler24.03-lts镜像,MindIE 服务启动失败的核心原因是 Qwen3-32B 模型未被 ATB 框架原生支持,求助怎么解决?

[root@localhost bin]# cd /usr/local/Ascend/mindie/latest/mindie-service/bin ./mindieservice_daemon The old environment variable ATB_LOG_TO_STDOUT will be deprecated on 2025/12/31. Please use the new environment variable MINDIE_LOG_TO_STDOUT as soon as possible. The old environment variable ATB_LOG_LEVEL will be deprecated on 2025/12/31. Please use the new environment variable MINDIE_LOG_LEVEL as soon as possible. The old environment variable MINDIE_LLM_LOG_LEVEL will be deprecated on 2025/12/31. Please use the new environment variable MINDIE_LOG_LEVEL as soon as possible. The old environment variable MINDIE_LLM_PYTHON_LOG_TO_STDOUT will be deprecated on 2025/12/31. Please use the new environment variable MINDIE_LOG_TO_STDOUT as soon as possible. The old environment variable MINDIE_LLM_LOG_TO_STDOUT will be deprecated on 2025/12/31. Please use the new environment variable MINDIE_LOG_TO_STDOUT as soon as possible. The old environment variable MINDIE_LLM_PYTHON_LOG_PATH will be deprecated on 2025/12/31. Please use the new environment variable MINDIE_LOG_PATH as soon as possible. The old environment variable ATB_LOG_TO_FILE will be deprecated on 2025/12/31. Please use the new environment variable MINDIE_LOG_TO_FILE as soon as possible. The old environment variable MINDIE_LLM_PYTHON_LOG_LEVEL will be deprecated on 2025/12/31. Please use the new environment variable MINDIE_LOG_LEVEL as soon as possible. The old environment variable MINDIE_LLM_PYTHON_LOG_TO_FILE will be deprecated on 2025/12/31. Please use the new environment variable MINDIE_LOG_TO_FILE as soon as possible. The old environment variable MINDIE_LLM_LOG_TO_FILE will be deprecated on 2025/12/31. Please use the new environment variable MINDIE_LOG_TO_FILE as soon as possible. The old environment variable OCK_LOG_LEVEL will be deprecated on 2025/12/31. Please use the new environment variable MINDIE_LOG_LEVEL as soon as possible. The old environment variable OCK_LOG_TO_STDOUT will be deprecated on 2025/12/31. Please use the new environment variable MINDIE_LOG_TO_STDOUT as soon as possible. 2025-06-25 03:33:39.884 1103 LLM log default format: [yyyy-mm-dd hh:mm:ss.uuuuuu] [processid] [threadid] [llm] [loglevel] [file:line] [status code] msg [2025-06-25 03:33:39.884] [1101] [281472814079968] [llm] [INFO] [llm_manager_impl.cpp:76] LLMRuntime init success! [2025-06-25 03:33:39.885] [1101] [281472814079968] [llm] [DEBUG] [model_deploy_config.cpp:80] ModelDeployConfig::pySonamelibpython3.11.so [2025-06-25 03:33:44.550] [1101] [281472814079968] [llm] [ERROR] [model_deploy_config.cpp:102] Failed to get vocab size from tokenizer wrapper with exception: NotImplementedError: unsupported model type: qwen3;请确认atb_llm.models路径下是否存在名为qwen3的文件夹。 At: /usr/local/Ascend/atb-models/atb_llm/models/__init__.py(94): get_model /usr/local/Ascend/atb-models/atb_llm/runner/tokenizer_wrapper.py(18): __init__ /usr/local/lib/python3.11/site-packages/mindie_llm/modeling/model_wrapper/atb/atb_tokenizer_wrapper.py(8): __init__ /usr/local/lib/python3.11/site-packages/mindie_llm/modeling/model_wrapper/__init__.py(28): get_tokenizer_wrapper [2025-06-25 03:33:44.550] [1101] [281472814079968] [llm] [ERROR] [llm_manager_impl.cpp:784] Config manager init exception: NotImplementedError: unsupported model type: qwen3;请确认atb_llm.models路径下是否存在名为qwen3的文件夹。 At: /usr/local/Ascend/atb-models/atb_llm/models/__init__.py(94): get_model /usr/local/Ascend/atb-models/atb_llm/runner/tokenizer_wrapper.py(18): __init__ /usr/local/lib/python3.11/site-packages/mindie_llm/modeling/model_wrapper/atb/atb_tokenizer_wrapper.py(8): __init__ /usr/local/lib/python3.11/site-packages/mindie_llm/modeling/model_wrapper/__init__.py(28): get_tokenizer_wrapper [2025-06-25 03:33:44.551+08:00] [1101] [1103] [server] [ERROR] [llm_infer_model_instance.cpp:284] : MIE05E040001[llm_backend] [LLMInferModelInstance::Init] llmManager_ init fail! LLMInferEngine failed to init LLMInferModels [2025-06-25 03:33:44.551+08:00] [1101] [1103] [server] [ERROR] [llm_infer_model.cpp:28] : MIE05E040001[llm_backend] ModelInstance init failed [2025-06-25 03:33:44.551+08:00] [1101] [1103] [server] [ERROR] [llm_infer_engine.cpp:41] : MIE05E040001[llm_backend] LLMInferEngine initialize LLMInferModels failed. please check the error stacks for more details [2025-06-25 03:33:44.551+08:00] [1101] [localhost.localdomain] [system] [server] [Start engine(mindieservice_llm_engine)] [fail] [2025-06-25 03:33:44.551+08:00] [1101] [1103] [server] [ERROR] [infer_backend_manager.cpp:216] : [MIE04E06011C] [infer_backend_manager] Failed to init engine. backendName is mindieservice_llm_engine [2025-06-25 03:33:44.551+08:00] [1101] [1103] [server] [ERROR] [endpoint.cpp:45] : [MIE04E02011C] [endpoint] Failed to init engine! Please check in the env MINDIE_LOG_PATH (default ~/mindie/log) or console output. [2025-06-25 03:33:44.551+08:00] [1101] [localhost.localdomain] [system] [server] [Start mindie server] [fail] Killed [root@localhost bin]#

我要发帖子