各位大神好：
我在我的在8卡910b上部署DeepseekV44 flash失败，报Value error, The checkpoint you are trying to load has model type `deepseek_v4` but Transformers does not recognize this architecture. This could be because of an issue with the checkpoint, or because your version of Transformers is out of date错误。
我是按照 vllm-ascend/docs/source/tutorials/DeepSeek-V4.md at releases/v0.13.0 · vllm-project/vllm-ascend 这篇文章一步一步来的，是错在哪里呢？
1.详细报错情况：
[root@localhost deepseekv4flashw8a8]# vllm serve /data/models/deepseekv4flashw8a8 \
--host 0.0.0.0 \
--max_model_len 524288 \
--max-num-batched-tokens 8192 \
--served-model-name ds \
--gpu-memory-utilization 0.9 \
--max-num-seqs 8 \
--data-parallel-size 1 \
--tensor-parallel-size 8 \
--enable-expert-parallel \
--quantization ascend \
--port 8006 \
--block-size 128 \
--chat-template /data/models/deepseekv4flashw8a8/chat_template.jinja \
--async-scheduling \
--additional-config '{"enable_cpu_binding": "true", "multistream_overlap_shared_expert": true}' \
--speculative-config '{"num_speculative_tokens": 1,"method": "deepseek_mtp"}' \
--compilation-config '{"cudagraph_mode":"FULL_DECODE_ONLY"}' \
--trust-remote-code
INFO 04-29 01:03:14 [__init__.py:43] Available plugins for group vllm.platform_plugins:
INFO 04-29 01:03:14 [__init__.py:45] - ascend -> vllm_ascend:register
INFO 04-29 01:03:14 [__init__.py:48] All plugins in this group will be loaded. Set `VLLM_PLUGINS` to control which plugins to load.
INFO 04-29 01:03:14 [__init__.py:217] Platform plugin ascend is activated
INFO 04-29 01:03:21 [__init__.py:108] Registered model loader `<class 'vllm_ascend.model_loader.netloader.netloader.ModelNetLoaderElastic'>` with load format `netloader`
(APIServer pid=848) INFO 04-29 01:03:22 [api_server.py:1351] vLLM API server version 0.13.0
(APIServer pid=848) INFO 04-29 01:03:22 [utils.py:253] non-default args: {'model_tag': '/data/models/deepseekv4flashw8a8', 'host': '0.0.0.0', 'port': 8006, 'chat_template': '/data/models/deepseekv4flashw8a8/chat_template.jinja', 'model': '/data/models/deepseekv4flashw8a8', 'trust_remote_code': True, 'max_model_len': 524288, 'quantization': 'ascend', 'served_model_name': ['ds'], 'tensor_parallel_size': 8, 'enable_expert_parallel': True, 'block_size': 128, 'max_num_batched_tokens': 8192, 'max_num_seqs': 8, 'async_scheduling': True, 'speculative_config': {'num_speculative_tokens': 1, 'method': 'deepseek_mtp'}, 'compilation_config': {'level': None, 'mode': None, 'debug_dump_path': None, 'cache_dir': '', 'compile_cache_save_format': 'binary', 'backend': 'vllm_ascend.compilation.compiler_interface.AscendCompiler', 'custom_ops': [], 'splitting_ops': None, 'compile_mm_encoder': False, 'compile_sizes': None, 'compile_ranges_split_points': None, 'inductor_compile_config': {'enable_auto_functionalized_v2': False}, 'inductor_passes': {}, 'cudagraph_mode': <CUDAGraphMode.FULL_DECODE_ONLY: (2, 0)>, 'cudagraph_num_of_warmups': 0, 'cudagraph_capture_sizes': None, 'cudagraph_copy_inputs': False, 'cudagraph_specialize_lora': True, 'use_inductor_graph_partition': None, 'pass_config': {}, 'max_cudagraph_capture_size': None, 'dynamic_shapes_config': {'type': <DynamicShapesType.BACKED: 'backed'>, 'evaluate_guards': False}, 'local_cache_dir': None}, 'additional_config': {'enable_cpu_binding': 'true', 'multistream_overlap_shared_expert': True}}
(APIServer pid=848) The argument `trust_remote_code` is to be used with Auto classes. It has no effect here and is ignored.
(APIServer pid=848) Traceback (most recent call last):
(APIServer pid=848) File "/usr/local/python3.11.14/bin/vllm", line 6, in <module>
(APIServer pid=848) sys.exit(main())
(APIServer pid=848) ^^^^^^
(APIServer pid=848) File "/vllm-workspace/vllm/vllm/entrypoints/cli/main.py", line 73, in main
(APIServer pid=848) args.dispatch_function(args)
(APIServer pid=848) File "/vllm-workspace/vllm/vllm/entrypoints/cli/serve.py", line 60, in cmd
(APIServer pid=848) uvloop.run(run_server(args))
(APIServer pid=848) File "/usr/local/python3.11.14/lib/python3.11/site-packages/uvloop/__init__.py", line 92, in run
(APIServer pid=848) return runner.run(wrapper())
(APIServer pid=848) ^^^^^^^^^^^^^^^^^^^^^
(APIServer pid=848) File "/usr/local/python3.11.14/lib/python3.11/asyncio/runners.py", line 118, in run
(APIServer pid=848) return self._loop.run_until_complete(task)
(APIServer pid=848) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
(APIServer pid=848) File "uvloop/loop.pyx", line 1518, in uvloop.loop.Loop.run_until_complete
(APIServer pid=848) File "/usr/local/python3.11.14/lib/python3.11/site-packages/uvloop/__init__.py", line 48, in wrapper
(APIServer pid=848) return await main
(APIServer pid=848) ^^^^^^^^^^
(APIServer pid=848) File "/vllm-workspace/vllm/vllm/entrypoints/openai/api_server.py", line 1398, in run_server
(APIServer pid=848) await run_server_worker(listen_address, sock, args, **uvicorn_kwargs)
(APIServer pid=848) File "/vllm-workspace/vllm/vllm/entrypoints/openai/api_server.py", line 1417, in run_server_worker
(APIServer pid=848) async with build_async_engine_client(
(APIServer pid=848) File "/usr/local/python3.11.14/lib/python3.11/contextlib.py", line 210, in __aenter__
(APIServer pid=848) return await anext(self.gen)
(APIServer pid=848) ^^^^^^^^^^^^^^^^^^^^^
(APIServer pid=848) File "/vllm-workspace/vllm/vllm/entrypoints/openai/api_server.py", line 172, in build_async_engine_client
(APIServer pid=848) async with build_async_engine_client_from_engine_args(
(APIServer pid=848) File "/usr/local/python3.11.14/lib/python3.11/contextlib.py", line 210, in __aenter__
(APIServer pid=848) return await anext(self.gen)
(APIServer pid=848) ^^^^^^^^^^^^^^^^^^^^^
(APIServer pid=848) File "/vllm-workspace/vllm/vllm/entrypoints/openai/api_server.py", line 198, in build_async_engine_client_from_engine_args
(APIServer pid=848) vllm_config = engine_args.create_engine_config(usage_context=usage_context)
(APIServer pid=848) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
(APIServer pid=848) File "/vllm-workspace/vllm/vllm/engine/arg_utils.py", line 1332, in create_engine_config
(APIServer pid=848) model_config = self.create_model_config()
(APIServer pid=848) ^^^^^^^^^^^^^^^^^^^^^^^^^^
(APIServer pid=848) File "/vllm-workspace/vllm/vllm/engine/arg_utils.py", line 1189, in create_model_config
(APIServer pid=848) return ModelConfig(
(APIServer pid=848) ^^^^^^^^^^^^
(APIServer pid=848) File "/usr/local/python3.11.14/lib/python3.11/site-packages/pydantic/_internal/_dataclasses.py", line 121, in __init__
(APIServer pid=848) s.__pydantic_validator__.validate_python(ArgsKwargs(args, kwargs), self_instance=s)
(APIServer pid=848) pydantic_core._pydantic_core.ValidationError: 1 validation error for ModelConfig
(APIServer pid=848) Value error, The checkpoint you are trying to load has model type `deepseek_v4` but Transformers does not recognize this architecture. This could be because of an issue with the checkpoint, or because your version of Transformers is out of date.
(APIServer pid=848)
(APIServer pid=848) You can update Transformers with the command `pip install --upgrade transformers`. If this does not work, and the checkpoint is very new, then there may not be a release version that supports this model yet. In this case, you can get the most up-to-date code by installing Transformers from source with the command `pip install git+https://github.com/huggingface/transformers.git` [type=value_error, input_value=ArgsKwargs((), {'model': ...rocessor_plugin': None}), input_type=ArgsKwargs]
(APIServer pid=848) For further information visit https://errors.pydantic.dev/2.13/v/value_error
(APIServer pid=848) [ERROR] 2026-04-29-01:03:22 (PID:848, Device:-1, RankID:-1) ERR99999 UNKNOWN applicaiton exception
(APIServer pid=848) sys:1: DeprecationWarning: builtin type swigvarlink has no __module__ attribute
2.我的操作系统：
[root@localhost deepseekv4flashw8a8]# uname -a
Linux localhost.localdomain 5.10.0-182.0.0.95.oe2203sp3.aarch64 #1 SMP Sat Dec 30 13:16:24 CST 2023 aarch64 aarch64 aarch64 GNU/Linux
3.镜像：
(1)我下载的镜像，包含v0.13.0rc3和v0.13.0rc3-openeuler我都试过了
[root@localhost deepseekv4flashw8a8]# docker images |grep ascend
quay.io/ascend/vllm-ascend nightly-main-openeuler 9e2c3c168dd1 35 hours ago 16.3GB
quay.io/ascend/vllm-ascend v0.13.0rc3-openeuler dcdaada5a8a6 4 days ago 16GB
quay.io/ascend/vllm-ascend v0.13.0rc3 357814603706 4 days ago 15.7GB
quay.io/ascend/vllm-ascend latest 82c06547d88e 3 months ago 17.7GB
(2)我当前启动的镜像：当前用的是vllm-ascend:v0.13.0rc3-openeuler,当然vllm-ascend:v0.13.0rc3我也是测试过的，一样的错误
docker启动命令：
docker run -itd \
--name vllm13 \
--restart=always \
--net=host \
--privileged \
--shm-size=512g \
--device=/dev/davinci0 \
--device=/dev/davinci1 \
--device=/dev/davinci2 \
--device=/dev/davinci3 \
--device=/dev/davinci4 \
--device=/dev/davinci5 \
--device=/dev/davinci6 \
--device=/dev/davinci7 \
--device=/dev/davinci_manager \
--device=/dev/hisi_hdc \
--device=/dev/devmm_svm \
-v /usr/local/Ascend/driver:/usr/local/Ascend/driver:ro \
-v /usr/local/bin/npu-smi:/usr/local/bin/npu-smi:ro \
-v /var/log/npu/:/var/log/npu/ \
-v /data/models:/data/models \
-v /var/log/vllm:/var/log/vllm \
quay.io/ascend/vllm-ascend:v0.13.0rc3-openeuler
[root@localhost deepseekv4flashw8a8]# docker ps |grep vllm
efb78bf320a1 quay.io/ascend/vllm-ascend:v0.13.0rc3-openeuler "/bin/bash -c ' s…" 3 hours ago Up 3 hours vllm13
4.npu info信息
[root@localhost deepseekv4flashw8a8]# npu-smi info
+------------------------------------------------------------------------------------------------+
| npu-smi 25.5.0 Version: 25.5.0 |
+---------------------------+---------------+----------------------------------------------------+
| NPU Name | Health | Power(W) Temp(C) Hugepages-Usage(page)|
| Chip | Bus-Id | AICore(%) Memory-Usage(MB) HBM-Usage(MB) |
+===========================+===============+====================================================+
| 0 910B4-1 | OK | 96.9 40 0 / 0 |
| 0 | 0000:C1:00.0 | 0 0 / 0 3426 / 65536 |
+===========================+===============+====================================================+
| 1 910B4-1 | OK | 90.8 39 0 / 0 |
| 0 | 0000:C2:00.0 | 0 0 / 0 3426 / 65536 |
+===========================+===============+====================================================+
| 2 910B4-1 | OK | 87.7 39 0 / 0 |
| 0 | 0000:81:00.0 | 0 0 / 0 3426 / 65536 |
+===========================+===============+====================================================+
| 3 910B4-1 | OK | 89.3 39 0 / 0 |
| 0 | 0000:82:00.0 | 0 0 / 0 3426 / 65536 |
+===========================+===============+====================================================+
| 4 910B4-1 | OK | 91.9 45 0 / 0 |
| 0 | 0000:01:00.0 | 0 0 / 0 3426 / 65536 |
+===========================+===============+====================================================+
| 5 910B4-1 | OK | 91.8 44 0 / 0 |
| 0 | 0000:02:00.0 | 0 0 / 0 3426 / 65536 |
+===========================+===============+====================================================+
| 6 910B4-1 | OK | 92.5 44 0 / 0 |
| 0 | 0000:41:00.0 | 0 0 / 0 3426 / 65536 |
+===========================+===============+====================================================+
| 7 910B4-1 | OK | 88.5 44 0 / 0 |
| 0 | 0000:42:00.0 | 0 0 / 0 3425 / 65536 |
+===========================+===============+====================================================+
+---------------------------+---------------+----------------------------------------------------+
| NPU Chip | Process id | Process name | Process memory(MB) |
+===========================+===============+====================================================+
| No running processes found in NPU 0 |
+===========================+===============+====================================================+
| No running processes found in NPU 1 |
+===========================+===============+====================================================+
| No running processes found in NPU 2 |
+===========================+===============+====================================================+
| No running processes found in NPU 3 |
+===========================+===============+====================================================+
| No running processes found in NPU 4 |
+===========================+===============+====================================================+
| No running processes found in NPU 5 |
+===========================+===============+====================================================+
| No running processes found in NPU 6 |
+===========================+===============+====================================================+
| No running processes found in NPU 7 |
+===========================+===============+====================================================+
[root@localhost deepseekv4flashw8a8]#
5.下载的deepseek flash w8a8信息：
[root@localhost deepseekv4flashw8a8]# pwd
/data/models/deepseekv4flashw8a8
[root@localhost deepseekv4flashw8a8]# ls
chat_template.jinja quant_model_weights-00011-of-00070.safetensors quant_model_weights-00028-of-00070.safetensors quant_model_weights-00045-of-00070.safetensors quant_model_weights-00062-of-00070.safetensors
config.json quant_model_weights-00012-of-00070.safetensors quant_model_weights-00029-of-00070.safetensors quant_model_weights-00046-of-00070.safetensors quant_model_weights-00063-of-00070.safetensors
configuration.json quant_model_weights-00013-of-00070.safetensors quant_model_weights-00030-of-00070.safetensors quant_model_weights-00047-of-00070.safetensors quant_model_weights-00064-of-00070.safetensors
DeepSeek-V4_best_practice.yaml quant_model_weights-00014-of-00070.safetensors quant_model_weights-00031-of-00070.safetensors quant_model_weights-00048-of-00070.safetensors quant_model_weights-00065-of-00070.safetensors
generation_config.json quant_model_weights-00015-of-00070.safetensors quant_model_weights-00032-of-00070.safetensors quant_model_weights-00049-of-00070.safetensors quant_model_weights-00066-of-00070.safetensors
optional quant_model_weights-00016-of-00070.safetensors quant_model_weights-00033-of-00070.safetensors quant_model_weights-00050-of-00070.safetensors quant_model_weights-00067-of-00070.safetensors
quant_model_description.json quant_model_weights-00017-of-00070.safetensors quant_model_weights-00034-of-00070.safetensors quant_model_weights-00051-of-00070.safetensors quant_model_weights-00068-of-00070.safetensors
quant_model_weights-00001-of-00070.safetensors quant_model_weights-00018-of-00070.safetensors quant_model_weights-00035-of-00070.safetensors quant_model_weights-00052-of-00070.safetensors quant_model_weights-00069-of-00070.safetensors
quant_model_weights-00002-of-00070.safetensors quant_model_weights-00019-of-00070.safetensors quant_model_weights-00036-of-00070.safetensors quant_model_weights-00053-of-00070.safetensors quant_model_weights-00070-of-00070.safetensors
quant_model_weights-00003-of-00070.safetensors quant_model_weights-00020-of-00070.safetensors quant_model_weights-00037-of-00070.safetensors quant_model_weights-00054-of-00070.safetensors quant_model_weights.safetensors.index.json
quant_model_weights-00004-of-00070.safetensors quant_model_weights-00021-of-00070.safetensors quant_model_weights-00038-of-00070.safetensors quant_model_weights-00055-of-00070.safetensors README.md
quant_model_weights-00005-of-00070.safetensors quant_model_weights-00022-of-00070.safetensors quant_model_weights-00039-of-00070.safetensors quant_model_weights-00056-of-00070.safetensors tokenizer_config.json
quant_model_weights-00006-of-00070.safetensors quant_model_weights-00023-of-00070.safetensors quant_model_weights-00040-of-00070.safetensors quant_model_weights-00057-of-00070.safetensors tokenizer.json
quant_model_weights-00007-of-00070.safetensors quant_model_weights-00024-of-00070.safetensors quant_model_weights-00041-of-00070.safetensors quant_model_weights-00058-of-00070.safetensors
quant_model_weights-00008-of-00070.safetensors quant_model_weights-00025-of-00070.safetensors quant_model_weights-00042-of-00070.safetensors quant_model_weights-00059-of-00070.safetensors
quant_model_weights-00009-of-00070.safetensors quant_model_weights-00026-of-00070.safetensors quant_model_weights-00043-of-00070.safetensors quant_model_weights-00060-of-00070.safetensors
quant_model_weights-00010-of-00070.safetensors quant_model_weights-00027-of-00070.safetensors quant_model_weights-00044-of-00070.safetensors quant_model_weights-00061-of-00070.safetensors
[root@localhost deepseekv4flashw8a8]#
6.反复试过很多次，多个镜像版本我都拉下来试过，不知道是哪里出了问题？
求各位大神解惑
点赞
收藏
回复
分享
举报
已解决
各位大神好：
我在我的在8卡910b上部署DeepseekV44 flash失败，报Value error, The checkpoint you are trying to load has model type `deepseek_v4` but Transformers does not recognize this architecture. This could be because of an issue with the checkpoint, or because your version of Transformers is out of date错误。
我是按照 vllm-ascend/docs/source/tutorials/DeepSeek-V4.md at releases/v0.13.0 · vllm-project/vllm-ascend 这篇文章一步一步来的，是错在哪里呢？
1.详细报错情况：
[root@localhost deepseekv4flashw8a8]# vllm serve /data/models/deepseekv4flashw8a8 \
--host 0.0.0.0 \
--max_model_len 524288 \
--max-num-batched-tokens 8192 \
--served-model-name ds \
--gpu-memory-utilization 0.9 \
--max-num-seqs 8 \
--data-parallel-size 1 \
--tensor-parallel-size 8 \
--enable-expert-parallel \
--quantization ascend \
--port 8006 \
--block-size 128 \
--chat-template /data/models/deepseekv4flashw8a8/chat_template.jinja \
--async-scheduling \
--additional-config '{"enable_cpu_binding": "true", "multistream_overlap_shared_expert": true}' \
--speculative-config '{"num_speculative_tokens": 1,"method": "deepseek_mtp"}' \
--compilation-config '{"cudagraph_mode":"FULL_DECODE_ONLY"}' \
--trust-remote-code
INFO 04-29 01:03:14 [__init__.py:43] Available plugins for group vllm.platform_plugins:
INFO 04-29 01:03:14 [__init__.py:45] - ascend -> vllm_ascend:register
INFO 04-29 01:03:14 [__init__.py:48] All plugins in this group will be loaded. Set `VLLM_PLUGINS` to control which plugins to load.
INFO 04-29 01:03:14 [__init__.py:217] Platform plugin ascend is activated
INFO 04-29 01:03:21 [__init__.py:108] Registered model loader `<class 'vllm_ascend.model_loader.netloader.netloader.ModelNetLoaderElastic'>` with load format `netloader`
(APIServer pid=848) INFO 04-29 01:03:22 [api_server.py:1351] vLLM API server version 0.13.0
(APIServer pid=848) INFO 04-29 01:03:22 [utils.py:253] non-default args: {'model_tag': '/data/models/deepseekv4flashw8a8', 'host': '0.0.0.0', 'port': 8006, 'chat_template': '/data/models/deepseekv4flashw8a8/chat_template.jinja', 'model': '/data/models/deepseekv4flashw8a8', 'trust_remote_code': True, 'max_model_len': 524288, 'quantization': 'ascend', 'served_model_name': ['ds'], 'tensor_parallel_size': 8, 'enable_expert_parallel': True, 'block_size': 128, 'max_num_batched_tokens': 8192, 'max_num_seqs': 8, 'async_scheduling': True, 'speculative_config': {'num_speculative_tokens': 1, 'method': 'deepseek_mtp'}, 'compilation_config': {'level': None, 'mode': None, 'debug_dump_path': None, 'cache_dir': '', 'compile_cache_save_format': 'binary', 'backend': 'vllm_ascend.compilation.compiler_interface.AscendCompiler', 'custom_ops': [], 'splitting_ops': None, 'compile_mm_encoder': False, 'compile_sizes': None, 'compile_ranges_split_points': None, 'inductor_compile_config': {'enable_auto_functionalized_v2': False}, 'inductor_passes': {}, 'cudagraph_mode': <CUDAGraphMode.FULL_DECODE_ONLY: (2, 0)>, 'cudagraph_num_of_warmups': 0, 'cudagraph_capture_sizes': None, 'cudagraph_copy_inputs': False, 'cudagraph_specialize_lora': True, 'use_inductor_graph_partition': None, 'pass_config': {}, 'max_cudagraph_capture_size': None, 'dynamic_shapes_config': {'type': <DynamicShapesType.BACKED: 'backed'>, 'evaluate_guards': False}, 'local_cache_dir': None}, 'additional_config': {'enable_cpu_binding': 'true', 'multistream_overlap_shared_expert': True}}
(APIServer pid=848) The argument `trust_remote_code` is to be used with Auto classes. It has no effect here and is ignored.
(APIServer pid=848) Traceback (most recent call last):
(APIServer pid=848) File "/usr/local/python3.11.14/bin/vllm", line 6, in <module>
(APIServer pid=848) sys.exit(main())
(APIServer pid=848) ^^^^^^
(APIServer pid=848) File "/vllm-workspace/vllm/vllm/entrypoints/cli/main.py", line 73, in main
(APIServer pid=848) args.dispatch_function(args)
(APIServer pid=848) File "/vllm-workspace/vllm/vllm/entrypoints/cli/serve.py", line 60, in cmd
(APIServer pid=848) uvloop.run(run_server(args))
(APIServer pid=848) File "/usr/local/python3.11.14/lib/python3.11/site-packages/uvloop/__init__.py", line 92, in run
(APIServer pid=848) return runner.run(wrapper())
(APIServer pid=848) ^^^^^^^^^^^^^^^^^^^^^
(APIServer pid=848) File "/usr/local/python3.11.14/lib/python3.11/asyncio/runners.py", line 118, in run
(APIServer pid=848) return self._loop.run_until_complete(task)
(APIServer pid=848) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
(APIServer pid=848) File "uvloop/loop.pyx", line 1518, in uvloop.loop.Loop.run_until_complete
(APIServer pid=848) File "/usr/local/python3.11.14/lib/python3.11/site-packages/uvloop/__init__.py", line 48, in wrapper
(APIServer pid=848) return await main
(APIServer pid=848) ^^^^^^^^^^
(APIServer pid=848) File "/vllm-workspace/vllm/vllm/entrypoints/openai/api_server.py", line 1398, in run_server
(APIServer pid=848) await run_server_worker(listen_address, sock, args, **uvicorn_kwargs)
(APIServer pid=848) File "/vllm-workspace/vllm/vllm/entrypoints/openai/api_server.py", line 1417, in run_server_worker
(APIServer pid=848) async with build_async_engine_client(
(APIServer pid=848) File "/usr/local/python3.11.14/lib/python3.11/contextlib.py", line 210, in __aenter__
(APIServer pid=848) return await anext(self.gen)
(APIServer pid=848) ^^^^^^^^^^^^^^^^^^^^^
(APIServer pid=848) File "/vllm-workspace/vllm/vllm/entrypoints/openai/api_server.py", line 172, in build_async_engine_client
(APIServer pid=848) async with build_async_engine_client_from_engine_args(
(APIServer pid=848) File "/usr/local/python3.11.14/lib/python3.11/contextlib.py", line 210, in __aenter__
(APIServer pid=848) return await anext(self.gen)
(APIServer pid=848) ^^^^^^^^^^^^^^^^^^^^^
(APIServer pid=848) File "/vllm-workspace/vllm/vllm/entrypoints/openai/api_server.py", line 198, in build_async_engine_client_from_engine_args
(APIServer pid=848) vllm_config = engine_args.create_engine_config(usage_context=usage_context)
(APIServer pid=848) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
(APIServer pid=848) File "/vllm-workspace/vllm/vllm/engine/arg_utils.py", line 1332, in create_engine_config
(APIServer pid=848) model_config = self.create_model_config()
(APIServer pid=848) ^^^^^^^^^^^^^^^^^^^^^^^^^^
(APIServer pid=848) File "/vllm-workspace/vllm/vllm/engine/arg_utils.py", line 1189, in create_model_config
(APIServer pid=848) return ModelConfig(
(APIServer pid=848) ^^^^^^^^^^^^
(APIServer pid=848) File "/usr/local/python3.11.14/lib/python3.11/site-packages/pydantic/_internal/_dataclasses.py", line 121, in __init__
(APIServer pid=848) s.__pydantic_validator__.validate_python(ArgsKwargs(args, kwargs), self_instance=s)
(APIServer pid=848) pydantic_core._pydantic_core.ValidationError: 1 validation error for ModelConfig
(APIServer pid=848) Value error, The checkpoint you are trying to load has model type `deepseek_v4` but Transformers does not recognize this architecture. This could be because of an issue with the checkpoint, or because your version of Transformers is out of date.
(APIServer pid=848)
(APIServer pid=848) You can update Transformers with the command `pip install --upgrade transformers`. If this does not work, and the checkpoint is very new, then there may not be a release version that supports this model yet. In this case, you can get the most up-to-date code by installing Transformers from source with the command `pip install git+https://github.com/huggingface/transformers.git` [type=value_error, input_value=ArgsKwargs((), {'model': ...rocessor_plugin': None}), input_type=ArgsKwargs]
(APIServer pid=848) For further information visit https://errors.pydantic.dev/2.13/v/value_error
(APIServer pid=848) [ERROR] 2026-04-29-01:03:22 (PID:848, Device:-1, RankID:-1) ERR99999 UNKNOWN applicaiton exception
(APIServer pid=848) sys:1: DeprecationWarning: builtin type swigvarlink has no __module__ attribute
2.我的操作系统：
[root@localhost deepseekv4flashw8a8]# uname -a
Linux localhost.localdomain 5.10.0-182.0.0.95.oe2203sp3.aarch64 #1 SMP Sat Dec 30 13:16:24 CST 2023 aarch64 aarch64 aarch64 GNU/Linux
3.镜像：
(1)我下载的镜像，包含v0.13.0rc3和v0.13.0rc3-openeuler我都试过了
[root@localhost deepseekv4flashw8a8]# docker images |grep ascend
quay.io/ascend/vllm-ascend nightly-main-openeuler 9e2c3c168dd1 35 hours ago 16.3GB
quay.io/ascend/vllm-ascend v0.13.0rc3-openeuler dcdaada5a8a6 4 days ago 16GB
quay.io/ascend/vllm-ascend v0.13.0rc3 357814603706 4 days ago 15.7GB
quay.io/ascend/vllm-ascend latest 82c06547d88e 3 months ago 17.7GB
(2)我当前启动的镜像：当前用的是vllm-ascend:v0.13.0rc3-openeuler,当然vllm-ascend:v0.13.0rc3我也是测试过的，一样的错误
docker启动命令：
docker run -itd \
--name vllm13 \
--restart=always \
--net=host \
--privileged \
--shm-size=512g \
--device=/dev/davinci0 \
--device=/dev/davinci1 \
--device=/dev/davinci2 \
--device=/dev/davinci3 \
--device=/dev/davinci4 \
--device=/dev/davinci5 \
--device=/dev/davinci6 \
--device=/dev/davinci7 \
--device=/dev/davinci_manager \
--device=/dev/hisi_hdc \
--device=/dev/devmm_svm \
-v /usr/local/Ascend/driver:/usr/local/Ascend/driver:ro \
-v /usr/local/bin/npu-smi:/usr/local/bin/npu-smi:ro \
-v /var/log/npu/:/var/log/npu/ \
-v /data/models:/data/models \
-v /var/log/vllm:/var/log/vllm \
quay.io/ascend/vllm-ascend:v0.13.0rc3-openeuler
[root@localhost deepseekv4flashw8a8]# docker ps |grep vllm
efb78bf320a1 quay.io/ascend/vllm-ascend:v0.13.0rc3-openeuler "/bin/bash -c ' s…" 3 hours ago Up 3 hours vllm13
4.npu info信息
[root@localhost deepseekv4flashw8a8]# npu-smi info
+------------------------------------------------------------------------------------------------+
| npu-smi 25.5.0 Version: 25.5.0 |
+---------------------------+---------------+----------------------------------------------------+
| NPU Name | Health | Power(W) Temp(C) Hugepages-Usage(page)|
| Chip | Bus-Id | AICore(%) Memory-Usage(MB) HBM-Usage(MB) |
+===========================+===============+====================================================+
| 0 910B4-1 | OK | 96.9 40 0 / 0 |
| 0 | 0000:C1:00.0 | 0 0 / 0 3426 / 65536 |
+===========================+===============+====================================================+
| 1 910B4-1 | OK | 90.8 39 0 / 0 |
| 0 | 0000:C2:00.0 | 0 0 / 0 3426 / 65536 |
+===========================+===============+====================================================+
| 2 910B4-1 | OK | 87.7 39 0 / 0 |
| 0 | 0000:81:00.0 | 0 0 / 0 3426 / 65536 |
+===========================+===============+====================================================+
| 3 910B4-1 | OK | 89.3 39 0 / 0 |
| 0 | 0000:82:00.0 | 0 0 / 0 3426 / 65536 |
+===========================+===============+====================================================+
| 4 910B4-1 | OK | 91.9 45 0 / 0 |
| 0 | 0000:01:00.0 | 0 0 / 0 3426 / 65536 |
+===========================+===============+====================================================+
| 5 910B4-1 | OK | 91.8 44 0 / 0 |
| 0 | 0000:02:00.0 | 0 0 / 0 3426 / 65536 |
+===========================+===============+====================================================+
| 6 910B4-1 | OK | 92.5 44 0 / 0 |
| 0 | 0000:41:00.0 | 0 0 / 0 3426 / 65536 |
+===========================+===============+====================================================+
| 7 910B4-1 | OK | 88.5 44 0 / 0 |
| 0 | 0000:42:00.0 | 0 0 / 0 3425 / 65536 |
+===========================+===============+====================================================+
+---------------------------+---------------+----------------------------------------------------+
| NPU Chip | Process id | Process name | Process memory(MB) |
+===========================+===============+====================================================+
| No running processes found in NPU 0 |
+===========================+===============+====================================================+
| No running processes found in NPU 1 |
+===========================+===============+====================================================+
| No running processes found in NPU 2 |
+===========================+===============+====================================================+
| No running processes found in NPU 3 |
+===========================+===============+====================================================+
| No running processes found in NPU 4 |
+===========================+===============+====================================================+
| No running processes found in NPU 5 |
+===========================+===============+====================================================+
| No running processes found in NPU 6 |
+===========================+===============+====================================================+
| No running processes found in NPU 7 |
+===========================+===============+====================================================+
[root@localhost deepseekv4flashw8a8]#
5.下载的deepseek flash w8a8信息：
[root@localhost deepseekv4flashw8a8]# pwd
/data/models/deepseekv4flashw8a8
[root@localhost deepseekv4flashw8a8]# ls
chat_template.jinja quant_model_weights-00011-of-00070.safetensors quant_model_weights-00028-of-00070.safetensors quant_model_weights-00045-of-00070.safetensors quant_model_weights-00062-of-00070.safetensors
config.json quant_model_weights-00012-of-00070.safetensors quant_model_weights-00029-of-00070.safetensors quant_model_weights-00046-of-00070.safetensors quant_model_weights-00063-of-00070.safetensors
configuration.json quant_model_weights-00013-of-00070.safetensors quant_model_weights-00030-of-00070.safetensors quant_model_weights-00047-of-00070.safetensors quant_model_weights-00064-of-00070.safetensors
DeepSeek-V4_best_practice.yaml quant_model_weights-00014-of-00070.safetensors quant_model_weights-00031-of-00070.safetensors quant_model_weights-00048-of-00070.safetensors quant_model_weights-00065-of-00070.safetensors
generation_config.json quant_model_weights-00015-of-00070.safetensors quant_model_weights-00032-of-00070.safetensors quant_model_weights-00049-of-00070.safetensors quant_model_weights-00066-of-00070.safetensors
optional quant_model_weights-00016-of-00070.safetensors quant_model_weights-00033-of-00070.safetensors quant_model_weights-00050-of-00070.safetensors quant_model_weights-00067-of-00070.safetensors
quant_model_description.json quant_model_weights-00017-of-00070.safetensors quant_model_weights-00034-of-00070.safetensors quant_model_weights-00051-of-00070.safetensors quant_model_weights-00068-of-00070.safetensors
quant_model_weights-00001-of-00070.safetensors quant_model_weights-00018-of-00070.safetensors quant_model_weights-00035-of-00070.safetensors quant_model_weights-00052-of-00070.safetensors quant_model_weights-00069-of-00070.safetensors
quant_model_weights-00002-of-00070.safetensors quant_model_weights-00019-of-00070.safetensors quant_model_weights-00036-of-00070.safetensors quant_model_weights-00053-of-00070.safetensors quant_model_weights-00070-of-00070.safetensors
quant_model_weights-00003-of-00070.safetensors quant_model_weights-00020-of-00070.safetensors quant_model_weights-00037-of-00070.safetensors quant_model_weights-00054-of-00070.safetensors quant_model_weights.safetensors.index.json
quant_model_weights-00004-of-00070.safetensors quant_model_weights-00021-of-00070.safetensors quant_model_weights-00038-of-00070.safetensors quant_model_weights-00055-of-00070.safetensors README.md
quant_model_weights-00005-of-00070.safetensors quant_model_weights-00022-of-00070.safetensors quant_model_weights-00039-of-00070.safetensors quant_model_weights-00056-of-00070.safetensors tokenizer_config.json
quant_model_weights-00006-of-00070.safetensors quant_model_weights-00023-of-00070.safetensors quant_model_weights-00040-of-00070.safetensors quant_model_weights-00057-of-00070.safetensors tokenizer.json
quant_model_weights-00007-of-00070.safetensors quant_model_weights-00024-of-00070.safetensors quant_model_weights-00041-of-00070.safetensors quant_model_weights-00058-of-00070.safetensors
quant_model_weights-00008-of-00070.safetensors quant_model_weights-00025-of-00070.safetensors quant_model_weights-00042-of-00070.safetensors quant_model_weights-00059-of-00070.safetensors
quant_model_weights-00009-of-00070.safetensors quant_model_weights-00026-of-00070.safetensors quant_model_weights-00043-of-00070.safetensors quant_model_weights-00060-of-00070.safetensors
quant_model_weights-00010-of-00070.safetensors quant_model_weights-00027-of-00070.safetensors quant_model_weights-00044-of-00070.safetensors quant_model_weights-00061-of-00070.safetensors
[root@localhost deepseekv4flashw8a8]#
6.反复试过很多次，多个镜像版本我都拉下来试过，不知道是哪里出了问题？
求各位大神解惑