配置:单台服务器,有8张910B的显卡
vllm-ascend镜像:vllm-ascend:v0.12.0rc1-openeuler 或者 v0.11.0rc2-openeuler
问题:vllm命令能拉起 Qwen3-Coder-480B-A35B-Instruct-AWQ 模型吗?是不是本身还不支持啊?
报错日志:
(APIServer pid=1198) File "/vllm-workspace/vllm/vllm/entrypoints/openai/api_server.py", line 183, in build_async_engine_client
(APIServer pid=1198) async with build_async_engine_client_from_engine_args(
(APIServer pid=1198) File "/usr/local/python3.11.13/lib/python3.11/contextlib.py", line 210, in __aenter__
(APIServer pid=1198) return await anext(self.gen)
(APIServer pid=1198) ^^^^^^^^^^^^^^^^^^^^^
(APIServer pid=1198) File "/vllm-workspace/vllm/vllm/entrypoints/openai/api_server.py", line 209, in build_async_engine_client_from_engine_args
(APIServer pid=1198) vllm_config = engine_args.create_engine_config(usage_context=usage_context)
(APIServer pid=1198) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
(APIServer pid=1198) File "/vllm-workspace/vllm/vllm/engine/arg_utils.py", line 1365, in create_engine_config
(APIServer pid=1198) model_config = self.create_model_config()
(APIServer pid=1198) ^^^^^^^^^^^^^^^^^^^^^^^^^^
(APIServer pid=1198) File "/vllm-workspace/vllm/vllm/engine/arg_utils.py", line 1221, in create_model_config
(APIServer pid=1198) return ModelConfig(
(APIServer pid=1198) ^^^^^^^^^^^^
(APIServer pid=1198) File "/usr/local/python3.11.13/lib/python3.11/site-packages/pydantic/_internal/_dataclasses.py", line 121, in __init__
(APIServer pid=1198) s.__pydantic_validator__.validate_python(ArgsKwargs(args, kwargs), self_instance=s)
(APIServer pid=1198) pydantic_core._pydantic_core.ValidationError: 1 validation error for ModelConfig
(APIServer pid=1198) Value error, Quantization method specified in the model config (awq) does not match the quantization method specified in the `quantization` argument (ascend). [type=value_error, input_value=ArgsKwargs((), {'model': ...rocessor_plugin': None}), input_type=ArgsKwargs]
(APIServer pid=1198) For further information visit https://errors.pydantic.dev/2.12/v/value_error
(APIServer pid=1198) [ERROR] 2025-12-25-02:36:23 (PID:1198, Device:-1, RankID:-1) ERR99999 UNKNOWN applicaiton exception
vllm-ascend镜像:vllm-ascend:v0.12.0rc1-openeuler 或者 v0.11.0rc2-openeuler
问题:vllm命令能拉起 Qwen3-Coder-480B-A35B-Instruct-AWQ 模型吗?是不是本身还不支持啊?
报错日志:
(APIServer pid=1198) File "/vllm-workspace/vllm/vllm/entrypoints/openai/api_server.py", line 183, in build_async_engine_client
(APIServer pid=1198) async with build_async_engine_client_from_engine_args(
(APIServer pid=1198) File "/usr/local/python3.11.13/lib/python3.11/contextlib.py", line 210, in __aenter__
(APIServer pid=1198) return await anext(self.gen)
(APIServer pid=1198) ^^^^^^^^^^^^^^^^^^^^^
(APIServer pid=1198) File "/vllm-workspace/vllm/vllm/entrypoints/openai/api_server.py", line 209, in build_async_engine_client_from_engine_args
(APIServer pid=1198) vllm_config = engine_args.create_engine_config(usage_context=usage_context)
(APIServer pid=1198) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
(APIServer pid=1198) File "/vllm-workspace/vllm/vllm/engine/arg_utils.py", line 1365, in create_engine_config
(APIServer pid=1198) model_config = self.create_model_config()
(APIServer pid=1198) ^^^^^^^^^^^^^^^^^^^^^^^^^^
(APIServer pid=1198) File "/vllm-workspace/vllm/vllm/engine/arg_utils.py", line 1221, in create_model_config
(APIServer pid=1198) return ModelConfig(
(APIServer pid=1198) ^^^^^^^^^^^^
(APIServer pid=1198) File "/usr/local/python3.11.13/lib/python3.11/site-packages/pydantic/_internal/_dataclasses.py", line 121, in __init__
(APIServer pid=1198) s.__pydantic_validator__.validate_python(ArgsKwargs(args, kwargs), self_instance=s)
(APIServer pid=1198) pydantic_core._pydantic_core.ValidationError: 1 validation error for ModelConfig
(APIServer pid=1198) Value error, Quantization method specified in the model config (awq) does not match the quantization method specified in the `quantization` argument (ascend). [type=value_error, input_value=ArgsKwargs((), {'model': ...rocessor_plugin': None}), input_type=ArgsKwargs]
(APIServer pid=1198) For further information visit https://errors.pydantic.dev/2.12/v/value_error
(APIServer pid=1198) [ERROR] 2025-12-25-02:36:23 (PID:1198, Device:-1, RankID:-1) ERR99999 UNKNOWN applicaiton exception