昇腾910B上，部署的Deepseek各个版本都OK，目前已经有一定的经验。
我们尝试部署阿里的QwQ-32B模型，该注意的点都注意的，可是依然启动不了。
[2025-03-18 15:54:47.563691] [951] [281457142853568] [llm] [INFO] [llm_manager_impl.cpp:70] LLMRuntime init success!
[2025-03-18 15:54:47.564994] [951] [281457142853568] [llm] [INFO] [model_deploy_config.cpp:130] ModelDeployConfig::pySonamelibpython3.11.so
[2025-03-18 15:54:52.220745] [951] [281457142853568] [llm] [ERROR] [model_deploy_config.cpp:152] Failed to get vocab size from tokenizer wrapper with exception: ValueError: safe_get_tokenizer_from_pretrained failed. Please check the input parameters model_path and kwargs. If the input parameters are valid and the required files exist in model_path, make sure the folder's owner has execute permission. Otherwise, please check the function stack for detailed exception information.
At:
/usr/local/Ascend/atb-models/atb_llm/models/base/model_utils.py(207): wrapper
/usr/local/Ascend/atb-models/atb_llm/models/qwen2/router_qwen2.py(84): get_tokenizer
/usr/local/Ascend/atb-models/atb_llm/models/base/router.py(151): tokenizer
/usr/local/Ascend/atb-models/atb_llm/runner/tokenizer_wrapper.py(20): __init__
/usr/local/lib/python3.11/site-packages/mindie_llm/modeling/model_wrapper/atb/atb_tokenizer_wrapper.py(8): __init__
/usr/local/lib/python3.11/site-packages/mindie_llm/modeling/model_wrapper/__init__.py(28): get_tokenizer_wrapper
LLMInferEngine failed to init LLMInferModels
[2025-03-18 15:54:52.221+08:00] [951] [huawei2] [system] [server] [start engine(mindieservice_llm_engine)] [fail]
[2025-03-18 15:54:52.221+08:00] [951] [952] [mindie-server] [ERROR] [infer_backend_manager.cpp:236] : [MIE04E06011C] [infer_backend_manager] Failed to init engine: mindieservice_llm_engine
[2025-03-18 15:54:52.221+08:00] [951] [952] [mindie-server] [ERROR] [endpoint.cpp:53] : [MIE04E02011C] [endpoint] Failed to init engine! Please check in the mindservice.log, pythonlog.log or console output.
[2025-03-18 15:54:52.221+08:00] [951] [huawei2] [system] [server] [start mindie server] [fail]
[2025-03-18 15:55:02.822+08:00] [951] [huawei2] [system] [server] [start endpoint] [fail]
[2025-03-18 15:55:02.823+08:00] [951] [huawei2] [system] [server] [start mindie server] [fail]
ERR: Failed to init endpoint! Please check the service log or console output.
[2025-03-18 15:55:02.823+08:00] [951] [952] [mindie-server] [INFO] [llm_daemon.cpp:142] : [daemon] ERR: Failed to init endpoint! Please check the service log or console output.
Killed
我想问下，华为官方有过部署成功的案例吗？A800T 8张910B(64G) ,谢谢~~
点赞
收藏
回复
分享
举报
已解决
昇腾910B上，部署的Deepseek各个版本都OK，目前已经有一定的经验。
我们尝试部署阿里的QwQ-32B模型，该注意的点都注意的，可是依然启动不了。
[2025-03-18 15:54:47.563691] [951] [281457142853568] [llm] [INFO] [llm_manager_impl.cpp:70] LLMRuntime init success!
[2025-03-18 15:54:47.564994] [951] [281457142853568] [llm] [INFO] [model_deploy_config.cpp:130] ModelDeployConfig::pySonamelibpython3.11.so
[2025-03-18 15:54:52.220745] [951] [281457142853568] [llm] [ERROR] [model_deploy_config.cpp:152] Failed to get vocab size from tokenizer wrapper with exception: ValueError: safe_get_tokenizer_from_pretrained failed. Please check the input parameters model_path and kwargs. If the input parameters are valid and the required files exist in model_path, make sure the folder's owner has execute permission. Otherwise, please check the function stack for detailed exception information.
At:
/usr/local/Ascend/atb-models/atb_llm/models/base/model_utils.py(207): wrapper
/usr/local/Ascend/atb-models/atb_llm/models/qwen2/router_qwen2.py(84): get_tokenizer
/usr/local/Ascend/atb-models/atb_llm/models/base/router.py(151): tokenizer
/usr/local/Ascend/atb-models/atb_llm/runner/tokenizer_wrapper.py(20): __init__
/usr/local/lib/python3.11/site-packages/mindie_llm/modeling/model_wrapper/atb/atb_tokenizer_wrapper.py(8): __init__
/usr/local/lib/python3.11/site-packages/mindie_llm/modeling/model_wrapper/__init__.py(28): get_tokenizer_wrapper
LLMInferEngine failed to init LLMInferModels
[2025-03-18 15:54:52.221+08:00] [951] [huawei2] [system] [server] [start engine(mindieservice_llm_engine)] [fail]
[2025-03-18 15:54:52.221+08:00] [951] [952] [mindie-server] [ERROR] [infer_backend_manager.cpp:236] : [MIE04E06011C] [infer_backend_manager] Failed to init engine: mindieservice_llm_engine
[2025-03-18 15:54:52.221+08:00] [951] [952] [mindie-server] [ERROR] [endpoint.cpp:53] : [MIE04E02011C] [endpoint] Failed to init engine! Please check in the mindservice.log, pythonlog.log or console output.
[2025-03-18 15:54:52.221+08:00] [951] [huawei2] [system] [server] [start mindie server] [fail]
[2025-03-18 15:55:02.822+08:00] [951] [huawei2] [system] [server] [start endpoint] [fail]
[2025-03-18 15:55:02.823+08:00] [951] [huawei2] [system] [server] [start mindie server] [fail]
ERR: Failed to init endpoint! Please check the service log or console output.
[2025-03-18 15:55:02.823+08:00] [951] [952] [mindie-server] [INFO] [llm_daemon.cpp:142] : [daemon] ERR: Failed to init endpoint! Please check the service log or console output.
Killed
我想问下，华为官方有过部署成功的案例吗？A800T 8张910B(64G) ,谢谢~~