root@52d67b4a2912:/workspace# VLLM_USE_MODELSCOPE=True vllm serve Qwen/Qwen2.5-0.5B-Instruct --gpu-memory-utilization=0.4 --port 8123
INFO 01-07 06:25:41 [__init__.py:36] Available plugins for group vllm.platform_plugins:
INFO 01-07 06:25:41 [__init__.py:38] - ascend -> vllm_ascend:register
INFO 01-07 06:25:41 [__init__.py:41] All plugins in this group will be loaded. Set `VLLM_PLUGINS` to control which plugins to load.
INFO 01-07 06:25:42 [__init__.py:207] Platform plugin ascend is activated
WARNING 01-07 06:25:48 [_custom_ops.py:20] Failed to import from vllm._C with ModuleNotFoundError("No module named 'vllm._C'")
WARNING 01-07 06:25:49 [registry.py:582] Model architecture Qwen2VLForConditionalGeneration is already registered, and will be overwritten by the new model class vllm_ascend.models.qwen2_vl:AscendQwen2VLForConditionalGeneration.
WARNING 01-07 06:25:49 [registry.py:582] Model architecture Qwen3VLMoeForConditionalGeneration is already registered, and will be overwritten by the new model class vllm_ascend.models.qwen2_5_vl_without_padding:AscendQwen3VLMoeForConditionalGeneration.
WARNING 01-07 06:25:49 [registry.py:582] Model architecture Qwen3VLForConditionalGeneration is already registered, and will be overwritten by the new model class vllm_ascend.models.qwen2_5_vl_without_padding:AscendQwen3VLForConditionalGeneration.
WARNING 01-07 06:25:49 [registry.py:582] Model architecture Qwen2_5_VLForConditionalGeneration is already registered, and will be overwritten by the new model class vllm_ascend.models.qwen2_5_vl:AscendQwen2_5_VLForConditionalGeneration.
WARNING 01-07 06:25:49 [registry.py:582] Model architecture Qwen2_5OmniModel is already registered, and will be overwritten by the new model class vllm_ascend.models.qwen2_5_omni_thinker:AscendQwen2_5OmniThinkerForConditionalGeneration.
WARNING 01-07 06:25:49 [registry.py:582] Model architecture DeepseekV32ForCausalLM is already registered, and will be overwritten by the new model class vllm_ascend.models.deepseek_v3_2:CustomDeepseekV3ForCausalLM.
WARNING 01-07 06:25:49 [registry.py:582] Model architecture Qwen3NextForCausalLM is already registered, and will be overwritten by the new model class vllm_ascend.models.qwen3_next:CustomQwen3NextForCausalLM.
INFO 01-07 06:25:49 [importing.py:63] Triton not installed or not compatible; certain GPU-related functions will not be available.
(APIServer pid=963) INFO 01-07 06:25:49 [api_server.py:1839] vLLM API server version 0.11.0
(APIServer pid=963) INFO 01-07 06:25:49 [utils.py:233] non-default args: {'model_tag': 'Qwen/Qwen2.5-0.5B-Instruct', 'port': 8123, 'model': 'Qwen/Qwen2.5-0.5B-Instruct', 'gpu_memory_utilization': 0.4}
(APIServer pid=963) Downloading Model from https://www.modelscope.cn to directory: /root/.cache/modelscope/hub/models/Qwen/Qwen2.5-0.5B-Instruct
(APIServer pid=963) 2026-01-07 06:25:50,555 - modelscope - INFO - Target directory already exists, skipping creation.
(APIServer pid=963) Downloading Model from https://www.modelscope.cn to directory: /root/.cache/modelscope/hub/models/Qwen/Qwen2.5-0.5B-Instruct
(APIServer pid=963) 2026-01-07 06:25:52,074 - modelscope - INFO - Target directory already exists, skipping creation.
(APIServer pid=963) Downloading Model from https://www.modelscope.cn to directory: /root/.cache/modelscope/hub/models/Qwen/Qwen2.5-0.5B-Instruct
(APIServer pid=963) 2026-01-07 06:25:53,272 - modelscope - INFO - Target directory already exists, skipping creation.
(APIServer pid=963) INFO 01-07 06:25:53 [model.py:547] Resolved architecture: Qwen2ForCausalLM
(APIServer pid=963) `torch_dtype` is deprecated! Use `dtype` instead!
(APIServer pid=963) INFO 01-07 06:25:53 [model.py:1510] Using max model len 32768
(APIServer pid=963) INFO 01-07 06:25:53 [scheduler.py:205] Chunked prefill is enabled with max_num_batched_tokens=2048.
(APIServer pid=963) INFO 01-07 06:25:53 [platform.py:213] PIECEWISE compilation enabled on NPU. use_inductor not supported - using only ACL Graph mode
(APIServer pid=963) INFO 01-07 06:25:53 [utils.py:450] Calculated maximum supported batch sizes for ACL graph: 72
(APIServer pid=963) WARNING 01-07 06:25:53 [utils.py:453] Currently, communication is performed using FFTS+ method, which reduces the number of available streams and, as a result, limits the range of runtime shapes that can be handled. To both improve communication performance and increase the number of supported shapes, set HCCL_OP_EXPANSION_MODE=AIV.
(APIServer pid=963) INFO 01-07 06:25:53 [utils.py:483] No adjustment needed for ACL graph batch sizes: Qwen2ForCausalLM model (layers: 24) with 67 sizes
(APIServer pid=963) WARNING 01-07 06:25:53 [platform.py:275] If chunked prefill or prefix caching is enabled, block size must be set to 128.
(APIServer pid=963) Downloading Model from https://www.modelscope.cn to directory: /root/.cache/modelscope/hub/models/Qwen/Qwen2.5-0.5B-Instruct
(APIServer pid=963) 2026-01-07 06:25:55,155 - modelscope - INFO - Target directory already exists, skipping creation.
(APIServer pid=963) Downloading Model from https://www.modelscope.cn to directory: /root/.cache/modelscope/hub/models/Qwen/Qwen2.5-0.5B-Instruct
(APIServer pid=963) 2026-01-07 06:25:56,851 - modelscope - INFO - Target directory already exists, skipping creation.
INFO 01-07 06:26:04 [__init__.py:36] Available plugins for group vllm.platform_plugins:
INFO 01-07 06:26:04 [__init__.py:38] - ascend -> vllm_ascend:register
INFO 01-07 06:26:04 [__init__.py:41] All plugins in this group will be loaded. Set `VLLM_PLUGINS` to control which plugins to load.
INFO 01-07 06:26:04 [__init__.py:207] Platform plugin ascend is activated
WARNING 01-07 06:26:11 [_custom_ops.py:20] Failed to import from vllm._C with ModuleNotFoundError("No module named 'vllm._C'")
(EngineCore_DP0 pid=1102) INFO 01-07 06:26:11 [core.py:644] Waiting for init message from front-end.
(EngineCore_DP0 pid=1102) WARNING 01-07 06:26:11 [registry.py:582] Model architecture Qwen2VLForConditionalGeneration is already registered, and will be overwritten by the new model class vllm_ascend.models.qwen2_vl:AscendQwen2VLForConditionalGeneration.
(EngineCore_DP0 pid=1102) WARNING 01-07 06:26:11 [registry.py:582] Model architecture Qwen3VLMoeForConditionalGeneration is already registered, and will be overwritten by the new model class vllm_ascend.models.qwen2_5_vl_without_padding:AscendQwen3VLMoeForConditionalGeneration.
(EngineCore_DP0 pid=1102) WARNING 01-07 06:26:11 [registry.py:582] Model architecture Qwen3VLForConditionalGeneration is already registered, and will be overwritten by the new model class vllm_ascend.models.qwen2_5_vl_without_padding:AscendQwen3VLForConditionalGeneration.
(EngineCore_DP0 pid=1102) WARNING 01-07 06:26:11 [registry.py:582] Model architecture Qwen2_5_VLForConditionalGeneration is already registered, and will be overwritten by the new model class vllm_ascend.models.qwen2_5_vl:AscendQwen2_5_VLForConditionalGeneration.
(EngineCore_DP0 pid=1102) WARNING 01-07 06:26:11 [registry.py:582] Model architecture Qwen2_5OmniModel is already registered, and will be overwritten by the new model class vllm_ascend.models.qwen2_5_omni_thinker:AscendQwen2_5OmniThinkerForConditionalGeneration.
(EngineCore_DP0 pid=1102) WARNING 01-07 06:26:11 [registry.py:582] Model architecture DeepseekV32ForCausalLM is already registered, and will be overwritten by the new model class vllm_ascend.models.deepseek_v3_2:CustomDeepseekV3ForCausalLM.
(EngineCore_DP0 pid=1102) WARNING 01-07 06:26:11 [registry.py:582] Model architecture Qwen3NextForCausalLM is already registered, and will be overwritten by the new model class vllm_ascend.models.qwen3_next:CustomQwen3NextForCausalLM.
(EngineCore_DP0 pid=1102) INFO 01-07 06:26:11 [core.py:77] Initializing a V1 LLM engine (v0.11.0) with config: model='Qwen/Qwen2.5-0.5B-Instruct', speculative_config=None, tokenizer='Qwen/Qwen2.5-0.5B-Instruct', skip_tokenizer_init=False, tokenizer_mode=auto, revision=None, tokenizer_revision=None, trust_remote_code=False, dtype=torch.bfloat16, max_seq_len=32768, download_dir=None, load_format=auto, tensor_parallel_size=1, pipeline_parallel_size=1, data_parallel_size=1, disable_custom_all_reduce=True, quantization=None, enforce_eager=False, kv_cache_dtype=auto, device_config=npu, structured_outputs_config=StructuredOutputsConfig(backend='auto', disable_fallback=False, disable_any_whitespace=False, disable_additional_properties=False, reasoning_parser=''), observability_config=ObservabilityConfig(show_hidden_metrics_for_version=None, otlp_traces_endpoint=None, collect_detailed_traces=None), seed=0, served_model_name=Qwen/Qwen2.5-0.5B-Instruct, enable_prefix_caching=True, chunked_prefill_enabled=True, pooler_config=None, compilation_config={"level":3,"debug_dump_path":"","cache_dir":"","backend":"","custom_ops":["all"],"splitting_ops":["vllm.unified_attention","vllm.unified_attention_with_output","vllm.mamba_mixer2","vllm.mamba_mixer","vllm.short_conv","vllm.linear_attention","vllm.plamo2_mamba_mixer","vllm.gdn_attention","vllm.sparse_attn_indexer","vllm.unified_ascend_attention_with_output","vllm.mla_forward"],"use_inductor":false,"compile_sizes":[],"inductor_compile_config":{"enable_auto_functionalized_v2":false},"inductor_passes":{},"cudagraph_mode":1,"use_cudagraph":true,"cudagraph_num_of_warmups":1,"cudagraph_capture_sizes":[512,504,496,488,480,472,464,456,448,440,432,424,416,408,400,392,384,376,368,360,352,344,336,328,320,312,304,296,288,280,272,264,256,248,240,232,224,216,208,200,192,184,176,168,160,152,144,136,128,120,112,104,96,88,80,72,64,56,48,40,32,24,16,8,4,2,1],"cudagraph_copy_inputs":false,"full_cuda_graph":false,"use_inductor_graph_partition":false,"pass_config":{},"max_capture_size":512,"local_cache_dir":null}
(EngineCore_DP0 pid=1102) INFO 01-07 06:26:11 [importing.py:63] Triton not installed or not compatible; certain GPU-related functions will not be available.
(EngineCore_DP0 pid=1102) INFO 01-07 06:26:13 [parallel_state.py:1208] rank 0 in world size 1 is assigned as DP rank 0, PP rank 0, TP rank 0, EP rank 0
(EngineCore_DP0 pid=1102) INFO 01-07 06:26:14 [model_runner_v1.py:2681] Starting to load model Qwen/Qwen2.5-0.5B-Instruct...
(EngineCore_DP0 pid=1102) Downloading Model from https://www.modelscope.cn to directory: /root/.cache/modelscope/hub/models/Qwen/Qwen2.5-0.5B-Instruct
(EngineCore_DP0 pid=1102) 2026-01-07 06:26:15,977 - modelscope - INFO - Target directory already exists, skipping creation.
Loading safetensors checkpoint shards: 0% Completed | 0/1 [00:00<?, ?it/s]
Loading safetensors checkpoint shards: 100% Completed | 1/1 [00:00<00:00, 4.31it/s]
Loading safetensors checkpoint shards: 100% Completed | 1/1 [00:00<00:00, 4.31it/s]
(EngineCore_DP0 pid=1102)
(EngineCore_DP0 pid=1102) INFO 01-07 06:26:16 [default_loader.py:267] Loading weights took 0.29 seconds
(EngineCore_DP0 pid=1102) INFO 01-07 06:26:17 [model_runner_v1.py:2707] Loading model weights took 0.9275 GB
(EngineCore_DP0 pid=1102) WARNING 01-07 06:26:17 [cudagraph_dispatcher.py:106] cudagraph dispatching keys are not initialized. No cudagraph will be used.
(EngineCore_DP0 pid=1102) INFO 01-07 06:26:23 [backends.py:548] Using cache directory: /root/.cache/vllm/torch_compile_cache/cedb71c321/rank_0_0/backbone for vLLM's torch.compile
(EngineCore_DP0 pid=1102) INFO 01-07 06:26:23 [backends.py:559] Dynamo bytecode transform time: 5.86 s
(EngineCore_DP0 pid=1102) INFO 01-07 06:26:25 [backends.py:218] Compiling a graph for dynamic shape takes 1.45 s
(EngineCore_DP0 pid=1102) INFO 01-07 06:26:27 [monitor.py:34] torch.compile takes 7.32 s in total
(EngineCore_DP0 pid=1102) INFO 01-07 06:26:28 [worker_v1.py:256] Available memory: 0, total memory: 65452113920
(EngineCore_DP0 pid=1102) ERROR 01-07 06:26:28 [patch_core.py:58] EngineCore failed to start.
(EngineCore_DP0 pid=1102) ERROR 01-07 06:26:28 [patch_core.py:58] Traceback (most recent call last):
(EngineCore_DP0 pid=1102) ERROR 01-07 06:26:28 [patch_core.py:58] File "/vllm-workspace/vllm-ascend/vllm_ascend/patch/platform/patch_core.py", line 49, in run_engine_core
(EngineCore_DP0 pid=1102) ERROR 01-07 06:26:28 [patch_core.py:58] engine_core = EngineCoreProc(*args, **kwargs)
(EngineCore_DP0 pid=1102) ERROR 01-07 06:26:28 [patch_core.py:58] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
(EngineCore_DP0 pid=1102) ERROR 01-07 06:26:28 [patch_core.py:58] File "/vllm-workspace/vllm/vllm/v1/engine/core.py", line 498, in __init__
(EngineCore_DP0 pid=1102) ERROR 01-07 06:26:28 [patch_core.py:58] super().__init__(vllm_config, executor_class, log_stats,
(EngineCore_DP0 pid=1102) ERROR 01-07 06:26:28 [patch_core.py:58] File "/vllm-workspace/vllm/vllm/v1/engine/core.py", line 92, in __init__
(EngineCore_DP0 pid=1102) ERROR 01-07 06:26:28 [patch_core.py:58] self._initialize_kv_caches(vllm_config)
(EngineCore_DP0 pid=1102) ERROR 01-07 06:26:28 [patch_core.py:58] File "/vllm-workspace/vllm/vllm/v1/engine/core.py", line 199, in _initialize_kv_caches
(EngineCore_DP0 pid=1102) ERROR 01-07 06:26:28 [patch_core.py:58] kv_cache_configs = get_kv_cache_configs(vllm_config, kv_cache_specs,
(EngineCore_DP0 pid=1102) ERROR 01-07 06:26:28 [patch_core.py:58] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
(EngineCore_DP0 pid=1102) ERROR 01-07 06:26:28 [patch_core.py:58] File "/vllm-workspace/vllm/vllm/v1/core/kv_cache_utils.py", line 1243, in get_kv_cache_configs
(EngineCore_DP0 pid=1102) ERROR 01-07 06:26:28 [patch_core.py:58] check_enough_kv_cache_memory(vllm_config, kv_cache_spec_one_worker,
(EngineCore_DP0 pid=1102) ERROR 01-07 06:26:28 [patch_core.py:58] File "/vllm-workspace/vllm/vllm/v1/core/kv_cache_utils.py", line 699, in check_enough_kv_cache_memory
(EngineCore_DP0 pid=1102) ERROR 01-07 06:26:28 [patch_core.py:58] raise ValueError("No available memory for the cache blocks. "
(EngineCore_DP0 pid=1102) ERROR 01-07 06:26:28 [patch_core.py:58] ValueError: No available memory for the cache blocks. Try increasing `gpu_memory_utilization` when initializing the engine.
(EngineCore_DP0 pid=1102) Process EngineCore_DP0:
(EngineCore_DP0 pid=1102) Traceback (most recent call last):
(EngineCore_DP0 pid=1102) File "/usr/local/python3.11.13/lib/python3.11/multiprocessing/process.py", line 314, in _bootstrap
(EngineCore_DP0 pid=1102) self.run()
(EngineCore_DP0 pid=1102) File "/usr/local/python3.11.13/lib/python3.11/multiprocessing/process.py", line 108, in run
(EngineCore_DP0 pid=1102) self._target(*self._args, **self._kwargs)
(EngineCore_DP0 pid=1102) File "/vllm-workspace/vllm-ascend/vllm_ascend/patch/platform/patch_core.py", line 62, in run_engine_core
(EngineCore_DP0 pid=1102) raise e
(EngineCore_DP0 pid=1102) File "/vllm-workspace/vllm-ascend/vllm_ascend/patch/platform/patch_core.py", line 49, in run_engine_core
(EngineCore_DP0 pid=1102) engine_core = EngineCoreProc(*args, **kwargs)
(EngineCore_DP0 pid=1102) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
(EngineCore_DP0 pid=1102) File "/vllm-workspace/vllm/vllm/v1/engine/core.py", line 498, in __init__
(EngineCore_DP0 pid=1102) super().__init__(vllm_config, executor_class, log_stats,
(EngineCore_DP0 pid=1102) File "/vllm-workspace/vllm/vllm/v1/engine/core.py", line 92, in __init__
(EngineCore_DP0 pid=1102) self._initialize_kv_caches(vllm_config)
(EngineCore_DP0 pid=1102) File "/vllm-workspace/vllm/vllm/v1/engine/core.py", line 199, in _initialize_kv_caches
(EngineCore_DP0 pid=1102) kv_cache_configs = get_kv_cache_configs(vllm_config, kv_cache_specs,
(EngineCore_DP0 pid=1102) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
(EngineCore_DP0 pid=1102) File "/vllm-workspace/vllm/vllm/v1/core/kv_cache_utils.py", line 1243, in get_kv_cache_configs
(EngineCore_DP0 pid=1102) check_enough_kv_cache_memory(vllm_config, kv_cache_spec_one_worker,
(EngineCore_DP0 pid=1102) File "/vllm-workspace/vllm/vllm/v1/core/kv_cache_utils.py", line 699, in check_enough_kv_cache_memory
(EngineCore_DP0 pid=1102) raise ValueError("No available memory for the cache blocks. "
(EngineCore_DP0 pid=1102) ValueError: No available memory for the cache blocks. Try increasing `gpu_memory_utilization` when initializing the engine.
(APIServer pid=963) Traceback (most recent call last):
(APIServer pid=963) File "/usr/local/python3.11.13/bin/vllm", line 7, in <module>
(APIServer pid=963) sys.exit(main())
(APIServer pid=963) ^^^^^^
(APIServer pid=963) File "/vllm-workspace/vllm/vllm/entrypoints/cli/main.py", line 54, in main
(APIServer pid=963) args.dispatch_function(args)
(APIServer pid=963) File "/vllm-workspace/vllm/vllm/entrypoints/cli/serve.py", line 57, in cmd
(APIServer pid=963) uvloop.run(run_server(args))
(APIServer pid=963) File "/usr/local/python3.11.13/lib/python3.11/site-packages/uvloop/__init__.py", line 92, in run
(APIServer pid=963) return runner.run(wrapper())
(APIServer pid=963) ^^^^^^^^^^^^^^^^^^^^^
(APIServer pid=963) File "/usr/local/python3.11.13/lib/python3.11/asyncio/runners.py", line 118, in run
(APIServer pid=963) return self._loop.run_until_complete(task)
(APIServer pid=963) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
(APIServer pid=963) File "uvloop/loop.pyx", line 1518, in uvloop.loop.Loop.run_until_complete
(APIServer pid=963) File "/usr/local/python3.11.13/lib/python3.11/site-packages/uvloop/__init__.py", line 48, in wrapper
(APIServer pid=963) return await main
(APIServer pid=963) ^^^^^^^^^^
(APIServer pid=963) File "/vllm-workspace/vllm/vllm/entrypoints/openai/api_server.py", line 1884, in run_server
(APIServer pid=963) await run_server_worker(listen_address, sock, args, **uvicorn_kwargs)
(APIServer pid=963) File "/vllm-workspace/vllm/vllm/entrypoints/openai/api_server.py", line 1902, in run_server_worker
(APIServer pid=963) async with build_async_engine_client(
(APIServer pid=963) File "/usr/local/python3.11.13/lib/python3.11/contextlib.py", line 210, in __aenter__
(APIServer pid=963) return await anext(self.gen)
(APIServer pid=963) ^^^^^^^^^^^^^^^^^^^^^
(APIServer pid=963) File "/vllm-workspace/vllm/vllm/entrypoints/openai/api_server.py", line 180, in build_async_engine_client
(APIServer pid=963) async with build_async_engine_client_from_engine_args(
(APIServer pid=963) File "/usr/local/python3.11.13/lib/python3.11/contextlib.py", line 210, in __aenter__
(APIServer pid=963) return await anext(self.gen)
(APIServer pid=963) ^^^^^^^^^^^^^^^^^^^^^
(APIServer pid=963) File "/vllm-workspace/vllm/vllm/entrypoints/openai/api_server.py", line 225, in build_async_engine_client_from_engine_args
(APIServer pid=963) async_llm = AsyncLLM.from_vllm_config(
(APIServer pid=963) ^^^^^^^^^^^^^^^^^^^^^^^^^^
(APIServer pid=963) File "/vllm-workspace/vllm/vllm/utils/__init__.py", line 1572, in inner
(APIServer pid=963) return fn(*args, **kwargs)
(APIServer pid=963) ^^^^^^^^^^^^^^^^^^^
(APIServer pid=963) File "/vllm-workspace/vllm/vllm/v1/engine/async_llm.py", line 207, in from_vllm_config
(APIServer pid=963) return cls(
(APIServer pid=963) ^^^^
(APIServer pid=963) File "/vllm-workspace/vllm/vllm/v1/engine/async_llm.py", line 134, in __init__
(APIServer pid=963) self.engine_core = EngineCoreClient.make_async_mp_client(
(APIServer pid=963) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
(APIServer pid=963) File "/vllm-workspace/vllm/vllm/v1/engine/core_client.py", line 102, in make_async_mp_client
(APIServer pid=963) return AsyncMPClient(*client_args)
(APIServer pid=963) ^^^^^^^^^^^^^^^^^^^^^^^^^^^
(APIServer pid=963) File "/vllm-workspace/vllm/vllm/v1/engine/core_client.py", line 769, in __init__
(APIServer pid=963) super().__init__(
(APIServer pid=963) File "/vllm-workspace/vllm/vllm/v1/engine/core_client.py", line 448, in __init__
(APIServer pid=963) with launch_core_engines(vllm_config, executor_class,
(APIServer pid=963) File "/usr/local/python3.11.13/lib/python3.11/contextlib.py", line 144, in __exit__
(APIServer pid=963) next(self.gen)
(APIServer pid=963) File "/vllm-workspace/vllm/vllm/v1/engine/utils.py", line 732, in launch_core_engines
(APIServer pid=963) wait_for_engine_startup(
(APIServer pid=963) File "/vllm-workspace/vllm/vllm/v1/engine/utils.py", line 785, in wait_for_engine_startup
(APIServer pid=963) raise RuntimeError("Engine core initialization failed. "
(APIServer pid=963) RuntimeError: Engine core initialization failed. See root cause above. Failed core proc(s): {}
(APIServer pid=963) [ERROR] 2026-01-07-06:26:34 (PID:963, Device:-1, RankID:-1) ERR99999 UNKNOWN applicaiton exception
点赞
收藏
回复
分享
举报
已解决
机器型号：
Atlas 900 RCK A2 Compute Node
显卡信息：
目的：我想在一张加速卡上通过vllm-ascend启动多个模型推理服务，以达到加速卡复用的目的。推理服务跑在容器中。
问题：启动第一个推理服务没问题，但是启动第二个推理服务的时候，使用已被占用的加速卡时程序异常退出（显存充足的情况下）
具体执行的操作如下：
首先启动vllm-ascend容器：
之后启动第一个推理服务：VLLM_USE_MODELSCOPE=True vllm serve Qwen/Qwen3-0.6B --gpu-memory-utilization=0.4 能够成功启动
启动第二个推理服务：VLLM_USE_MODELSCOPE=True vllm serve Qwen/Qwen2.5-0.5B-Instruct --gpu-memory-utilization=0.4 --port 8123
服务异常退出，报错无法分配到显存
root@52d67b4a2912:/workspace# VLLM_USE_MODELSCOPE=True vllm serve Qwen/Qwen2.5-0.5B-Instruct --gpu-memory-utilization=0.4 --port 8123 INFO 01-07 06:25:41 [__init__.py:36] Available plugins for group vllm.platform_plugins: INFO 01-07 06:25:41 [__init__.py:38] - ascend -> vllm_ascend:register INFO 01-07 06:25:41 [__init__.py:41] All plugins in this group will be loaded. Set `VLLM_PLUGINS` to control which plugins to load. INFO 01-07 06:25:42 [__init__.py:207] Platform plugin ascend is activated WARNING 01-07 06:25:48 [_custom_ops.py:20] Failed to import from vllm._C with ModuleNotFoundError("No module named 'vllm._C'") WARNING 01-07 06:25:49 [registry.py:582] Model architecture Qwen2VLForConditionalGeneration is already registered, and will be overwritten by the new model class vllm_ascend.models.qwen2_vl:AscendQwen2VLForConditionalGeneration. WARNING 01-07 06:25:49 [registry.py:582] Model architecture Qwen3VLMoeForConditionalGeneration is already registered, and will be overwritten by the new model class vllm_ascend.models.qwen2_5_vl_without_padding:AscendQwen3VLMoeForConditionalGeneration. WARNING 01-07 06:25:49 [registry.py:582] Model architecture Qwen3VLForConditionalGeneration is already registered, and will be overwritten by the new model class vllm_ascend.models.qwen2_5_vl_without_padding:AscendQwen3VLForConditionalGeneration. WARNING 01-07 06:25:49 [registry.py:582] Model architecture Qwen2_5_VLForConditionalGeneration is already registered, and will be overwritten by the new model class vllm_ascend.models.qwen2_5_vl:AscendQwen2_5_VLForConditionalGeneration. WARNING 01-07 06:25:49 [registry.py:582] Model architecture Qwen2_5OmniModel is already registered, and will be overwritten by the new model class vllm_ascend.models.qwen2_5_omni_thinker:AscendQwen2_5OmniThinkerForConditionalGeneration. WARNING 01-07 06:25:49 [registry.py:582] Model architecture DeepseekV32ForCausalLM is already registered, and will be overwritten by the new model class vllm_ascend.models.deepseek_v3_2:CustomDeepseekV3ForCausalLM. WARNING 01-07 06:25:49 [registry.py:582] Model architecture Qwen3NextForCausalLM is already registered, and will be overwritten by the new model class vllm_ascend.models.qwen3_next:CustomQwen3NextForCausalLM. INFO 01-07 06:25:49 [importing.py:63] Triton not installed or not compatible; certain GPU-related functions will not be available. (APIServer pid=963) INFO 01-07 06:25:49 [api_server.py:1839] vLLM API server version 0.11.0 (APIServer pid=963) INFO 01-07 06:25:49 [utils.py:233] non-default args: {'model_tag': 'Qwen/Qwen2.5-0.5B-Instruct', 'port': 8123, 'model': 'Qwen/Qwen2.5-0.5B-Instruct', 'gpu_memory_utilization': 0.4} (APIServer pid=963) Downloading Model from https://www.modelscope.cn to directory: /root/.cache/modelscope/hub/models/Qwen/Qwen2.5-0.5B-Instruct (APIServer pid=963) 2026-01-07 06:25:50,555 - modelscope - INFO - Target directory already exists, skipping creation. (APIServer pid=963) Downloading Model from https://www.modelscope.cn to directory: /root/.cache/modelscope/hub/models/Qwen/Qwen2.5-0.5B-Instruct (APIServer pid=963) 2026-01-07 06:25:52,074 - modelscope - INFO - Target directory already exists, skipping creation. (APIServer pid=963) Downloading Model from https://www.modelscope.cn to directory: /root/.cache/modelscope/hub/models/Qwen/Qwen2.5-0.5B-Instruct (APIServer pid=963) 2026-01-07 06:25:53,272 - modelscope - INFO - Target directory already exists, skipping creation. (APIServer pid=963) INFO 01-07 06:25:53 [model.py:547] Resolved architecture: Qwen2ForCausalLM (APIServer pid=963) `torch_dtype` is deprecated! Use `dtype` instead! (APIServer pid=963) INFO 01-07 06:25:53 [model.py:1510] Using max model len 32768 (APIServer pid=963) INFO 01-07 06:25:53 [scheduler.py:205] Chunked prefill is enabled with max_num_batched_tokens=2048. (APIServer pid=963) INFO 01-07 06:25:53 [platform.py:213] PIECEWISE compilation enabled on NPU. use_inductor not supported - using only ACL Graph mode (APIServer pid=963) INFO 01-07 06:25:53 [utils.py:450] Calculated maximum supported batch sizes for ACL graph: 72 (APIServer pid=963) WARNING 01-07 06:25:53 [utils.py:453] Currently, communication is performed using FFTS+ method, which reduces the number of available streams and, as a result, limits the range of runtime shapes that can be handled. To both improve communication performance and increase the number of supported shapes, set HCCL_OP_EXPANSION_MODE=AIV. (APIServer pid=963) INFO 01-07 06:25:53 [utils.py:483] No adjustment needed for ACL graph batch sizes: Qwen2ForCausalLM model (layers: 24) with 67 sizes (APIServer pid=963) WARNING 01-07 06:25:53 [platform.py:275] If chunked prefill or prefix caching is enabled, block size must be set to 128. (APIServer pid=963) Downloading Model from https://www.modelscope.cn to directory: /root/.cache/modelscope/hub/models/Qwen/Qwen2.5-0.5B-Instruct (APIServer pid=963) 2026-01-07 06:25:55,155 - modelscope - INFO - Target directory already exists, skipping creation. (APIServer pid=963) Downloading Model from https://www.modelscope.cn to directory: /root/.cache/modelscope/hub/models/Qwen/Qwen2.5-0.5B-Instruct (APIServer pid=963) 2026-01-07 06:25:56,851 - modelscope - INFO - Target directory already exists, skipping creation. INFO 01-07 06:26:04 [__init__.py:36] Available plugins for group vllm.platform_plugins: INFO 01-07 06:26:04 [__init__.py:38] - ascend -> vllm_ascend:register INFO 01-07 06:26:04 [__init__.py:41] All plugins in this group will be loaded. Set `VLLM_PLUGINS` to control which plugins to load. INFO 01-07 06:26:04 [__init__.py:207] Platform plugin ascend is activated WARNING 01-07 06:26:11 [_custom_ops.py:20] Failed to import from vllm._C with ModuleNotFoundError("No module named 'vllm._C'") (EngineCore_DP0 pid=1102) INFO 01-07 06:26:11 [core.py:644] Waiting for init message from front-end. (EngineCore_DP0 pid=1102) WARNING 01-07 06:26:11 [registry.py:582] Model architecture Qwen2VLForConditionalGeneration is already registered, and will be overwritten by the new model class vllm_ascend.models.qwen2_vl:AscendQwen2VLForConditionalGeneration. (EngineCore_DP0 pid=1102) WARNING 01-07 06:26:11 [registry.py:582] Model architecture Qwen3VLMoeForConditionalGeneration is already registered, and will be overwritten by the new model class vllm_ascend.models.qwen2_5_vl_without_padding:AscendQwen3VLMoeForConditionalGeneration. (EngineCore_DP0 pid=1102) WARNING 01-07 06:26:11 [registry.py:582] Model architecture Qwen3VLForConditionalGeneration is already registered, and will be overwritten by the new model class vllm_ascend.models.qwen2_5_vl_without_padding:AscendQwen3VLForConditionalGeneration. (EngineCore_DP0 pid=1102) WARNING 01-07 06:26:11 [registry.py:582] Model architecture Qwen2_5_VLForConditionalGeneration is already registered, and will be overwritten by the new model class vllm_ascend.models.qwen2_5_vl:AscendQwen2_5_VLForConditionalGeneration. (EngineCore_DP0 pid=1102) WARNING 01-07 06:26:11 [registry.py:582] Model architecture Qwen2_5OmniModel is already registered, and will be overwritten by the new model class vllm_ascend.models.qwen2_5_omni_thinker:AscendQwen2_5OmniThinkerForConditionalGeneration. (EngineCore_DP0 pid=1102) WARNING 01-07 06:26:11 [registry.py:582] Model architecture DeepseekV32ForCausalLM is already registered, and will be overwritten by the new model class vllm_ascend.models.deepseek_v3_2:CustomDeepseekV3ForCausalLM. (EngineCore_DP0 pid=1102) WARNING 01-07 06:26:11 [registry.py:582] Model architecture Qwen3NextForCausalLM is already registered, and will be overwritten by the new model class vllm_ascend.models.qwen3_next:CustomQwen3NextForCausalLM. (EngineCore_DP0 pid=1102) INFO 01-07 06:26:11 [core.py:77] Initializing a V1 LLM engine (v0.11.0) with config: model='Qwen/Qwen2.5-0.5B-Instruct', speculative_config=None, tokenizer='Qwen/Qwen2.5-0.5B-Instruct', skip_tokenizer_init=False, tokenizer_mode=auto, revision=None, tokenizer_revision=None, trust_remote_code=False, dtype=torch.bfloat16, max_seq_len=32768, download_dir=None, load_format=auto, tensor_parallel_size=1, pipeline_parallel_size=1, data_parallel_size=1, disable_custom_all_reduce=True, quantization=None, enforce_eager=False, kv_cache_dtype=auto, device_config=npu, structured_outputs_config=StructuredOutputsConfig(backend='auto', disable_fallback=False, disable_any_whitespace=False, disable_additional_properties=False, reasoning_parser=''), observability_config=ObservabilityConfig(show_hidden_metrics_for_version=None, otlp_traces_endpoint=None, collect_detailed_traces=None), seed=0, served_model_name=Qwen/Qwen2.5-0.5B-Instruct, enable_prefix_caching=True, chunked_prefill_enabled=True, pooler_config=None, compilation_config={"level":3,"debug_dump_path":"","cache_dir":"","backend":"","custom_ops":["all"],"splitting_ops":["vllm.unified_attention","vllm.unified_attention_with_output","vllm.mamba_mixer2","vllm.mamba_mixer","vllm.short_conv","vllm.linear_attention","vllm.plamo2_mamba_mixer","vllm.gdn_attention","vllm.sparse_attn_indexer","vllm.unified_ascend_attention_with_output","vllm.mla_forward"],"use_inductor":false,"compile_sizes":[],"inductor_compile_config":{"enable_auto_functionalized_v2":false},"inductor_passes":{},"cudagraph_mode":1,"use_cudagraph":true,"cudagraph_num_of_warmups":1,"cudagraph_capture_sizes":[512,504,496,488,480,472,464,456,448,440,432,424,416,408,400,392,384,376,368,360,352,344,336,328,320,312,304,296,288,280,272,264,256,248,240,232,224,216,208,200,192,184,176,168,160,152,144,136,128,120,112,104,96,88,80,72,64,56,48,40,32,24,16,8,4,2,1],"cudagraph_copy_inputs":false,"full_cuda_graph":false,"use_inductor_graph_partition":false,"pass_config":{},"max_capture_size":512,"local_cache_dir":null} (EngineCore_DP0 pid=1102) INFO 01-07 06:26:11 [importing.py:63] Triton not installed or not compatible; certain GPU-related functions will not be available. (EngineCore_DP0 pid=1102) INFO 01-07 06:26:13 [parallel_state.py:1208] rank 0 in world size 1 is assigned as DP rank 0, PP rank 0, TP rank 0, EP rank 0 (EngineCore_DP0 pid=1102) INFO 01-07 06:26:14 [model_runner_v1.py:2681] Starting to load model Qwen/Qwen2.5-0.5B-Instruct... (EngineCore_DP0 pid=1102) Downloading Model from https://www.modelscope.cn to directory: /root/.cache/modelscope/hub/models/Qwen/Qwen2.5-0.5B-Instruct (EngineCore_DP0 pid=1102) 2026-01-07 06:26:15,977 - modelscope - INFO - Target directory already exists, skipping creation. Loading safetensors checkpoint shards: 0% Completed | 0/1 [00:00<?, ?it/s] Loading safetensors checkpoint shards: 100% Completed | 1/1 [00:00<00:00, 4.31it/s] Loading safetensors checkpoint shards: 100% Completed | 1/1 [00:00<00:00, 4.31it/s] (EngineCore_DP0 pid=1102) (EngineCore_DP0 pid=1102) INFO 01-07 06:26:16 [default_loader.py:267] Loading weights took 0.29 seconds (EngineCore_DP0 pid=1102) INFO 01-07 06:26:17 [model_runner_v1.py:2707] Loading model weights took 0.9275 GB (EngineCore_DP0 pid=1102) WARNING 01-07 06:26:17 [cudagraph_dispatcher.py:106] cudagraph dispatching keys are not initialized. No cudagraph will be used. (EngineCore_DP0 pid=1102) INFO 01-07 06:26:23 [backends.py:548] Using cache directory: /root/.cache/vllm/torch_compile_cache/cedb71c321/rank_0_0/backbone for vLLM's torch.compile (EngineCore_DP0 pid=1102) INFO 01-07 06:26:23 [backends.py:559] Dynamo bytecode transform time: 5.86 s (EngineCore_DP0 pid=1102) INFO 01-07 06:26:25 [backends.py:218] Compiling a graph for dynamic shape takes 1.45 s (EngineCore_DP0 pid=1102) INFO 01-07 06:26:27 [monitor.py:34] torch.compile takes 7.32 s in total (EngineCore_DP0 pid=1102) INFO 01-07 06:26:28 [worker_v1.py:256] Available memory: 0, total memory: 65452113920 (EngineCore_DP0 pid=1102) ERROR 01-07 06:26:28 [patch_core.py:58] EngineCore failed to start. (EngineCore_DP0 pid=1102) ERROR 01-07 06:26:28 [patch_core.py:58] Traceback (most recent call last): (EngineCore_DP0 pid=1102) ERROR 01-07 06:26:28 [patch_core.py:58] File "/vllm-workspace/vllm-ascend/vllm_ascend/patch/platform/patch_core.py", line 49, in run_engine_core (EngineCore_DP0 pid=1102) ERROR 01-07 06:26:28 [patch_core.py:58] engine_core = EngineCoreProc(*args, **kwargs) (EngineCore_DP0 pid=1102) ERROR 01-07 06:26:28 [patch_core.py:58] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (EngineCore_DP0 pid=1102) ERROR 01-07 06:26:28 [patch_core.py:58] File "/vllm-workspace/vllm/vllm/v1/engine/core.py", line 498, in __init__ (EngineCore_DP0 pid=1102) ERROR 01-07 06:26:28 [patch_core.py:58] super().__init__(vllm_config, executor_class, log_stats, (EngineCore_DP0 pid=1102) ERROR 01-07 06:26:28 [patch_core.py:58] File "/vllm-workspace/vllm/vllm/v1/engine/core.py", line 92, in __init__ (EngineCore_DP0 pid=1102) ERROR 01-07 06:26:28 [patch_core.py:58] self._initialize_kv_caches(vllm_config) (EngineCore_DP0 pid=1102) ERROR 01-07 06:26:28 [patch_core.py:58] File "/vllm-workspace/vllm/vllm/v1/engine/core.py", line 199, in _initialize_kv_caches (EngineCore_DP0 pid=1102) ERROR 01-07 06:26:28 [patch_core.py:58] kv_cache_configs = get_kv_cache_configs(vllm_config, kv_cache_specs, (EngineCore_DP0 pid=1102) ERROR 01-07 06:26:28 [patch_core.py:58] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (EngineCore_DP0 pid=1102) ERROR 01-07 06:26:28 [patch_core.py:58] File "/vllm-workspace/vllm/vllm/v1/core/kv_cache_utils.py", line 1243, in get_kv_cache_configs (EngineCore_DP0 pid=1102) ERROR 01-07 06:26:28 [patch_core.py:58] check_enough_kv_cache_memory(vllm_config, kv_cache_spec_one_worker, (EngineCore_DP0 pid=1102) ERROR 01-07 06:26:28 [patch_core.py:58] File "/vllm-workspace/vllm/vllm/v1/core/kv_cache_utils.py", line 699, in check_enough_kv_cache_memory (EngineCore_DP0 pid=1102) ERROR 01-07 06:26:28 [patch_core.py:58] raise ValueError("No available memory for the cache blocks. " (EngineCore_DP0 pid=1102) ERROR 01-07 06:26:28 [patch_core.py:58] ValueError: No available memory for the cache blocks. Try increasing `gpu_memory_utilization` when initializing the engine. (EngineCore_DP0 pid=1102) Process EngineCore_DP0: (EngineCore_DP0 pid=1102) Traceback (most recent call last): (EngineCore_DP0 pid=1102) File "/usr/local/python3.11.13/lib/python3.11/multiprocessing/process.py", line 314, in _bootstrap (EngineCore_DP0 pid=1102) self.run() (EngineCore_DP0 pid=1102) File "/usr/local/python3.11.13/lib/python3.11/multiprocessing/process.py", line 108, in run (EngineCore_DP0 pid=1102) self._target(*self._args, **self._kwargs) (EngineCore_DP0 pid=1102) File "/vllm-workspace/vllm-ascend/vllm_ascend/patch/platform/patch_core.py", line 62, in run_engine_core (EngineCore_DP0 pid=1102) raise e (EngineCore_DP0 pid=1102) File "/vllm-workspace/vllm-ascend/vllm_ascend/patch/platform/patch_core.py", line 49, in run_engine_core (EngineCore_DP0 pid=1102) engine_core = EngineCoreProc(*args, **kwargs) (EngineCore_DP0 pid=1102) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (EngineCore_DP0 pid=1102) File "/vllm-workspace/vllm/vllm/v1/engine/core.py", line 498, in __init__ (EngineCore_DP0 pid=1102) super().__init__(vllm_config, executor_class, log_stats, (EngineCore_DP0 pid=1102) File "/vllm-workspace/vllm/vllm/v1/engine/core.py", line 92, in __init__ (EngineCore_DP0 pid=1102) self._initialize_kv_caches(vllm_config) (EngineCore_DP0 pid=1102) File "/vllm-workspace/vllm/vllm/v1/engine/core.py", line 199, in _initialize_kv_caches (EngineCore_DP0 pid=1102) kv_cache_configs = get_kv_cache_configs(vllm_config, kv_cache_specs, (EngineCore_DP0 pid=1102) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (EngineCore_DP0 pid=1102) File "/vllm-workspace/vllm/vllm/v1/core/kv_cache_utils.py", line 1243, in get_kv_cache_configs (EngineCore_DP0 pid=1102) check_enough_kv_cache_memory(vllm_config, kv_cache_spec_one_worker, (EngineCore_DP0 pid=1102) File "/vllm-workspace/vllm/vllm/v1/core/kv_cache_utils.py", line 699, in check_enough_kv_cache_memory (EngineCore_DP0 pid=1102) raise ValueError("No available memory for the cache blocks. " (EngineCore_DP0 pid=1102) ValueError: No available memory for the cache blocks. Try increasing `gpu_memory_utilization` when initializing the engine. (APIServer pid=963) Traceback (most recent call last): (APIServer pid=963) File "/usr/local/python3.11.13/bin/vllm", line 7, in <module> (APIServer pid=963) sys.exit(main()) (APIServer pid=963) ^^^^^^ (APIServer pid=963) File "/vllm-workspace/vllm/vllm/entrypoints/cli/main.py", line 54, in main (APIServer pid=963) args.dispatch_function(args) (APIServer pid=963) File "/vllm-workspace/vllm/vllm/entrypoints/cli/serve.py", line 57, in cmd (APIServer pid=963) uvloop.run(run_server(args)) (APIServer pid=963) File "/usr/local/python3.11.13/lib/python3.11/site-packages/uvloop/__init__.py", line 92, in run (APIServer pid=963) return runner.run(wrapper()) (APIServer pid=963) ^^^^^^^^^^^^^^^^^^^^^ (APIServer pid=963) File "/usr/local/python3.11.13/lib/python3.11/asyncio/runners.py", line 118, in run (APIServer pid=963) return self._loop.run_until_complete(task) (APIServer pid=963) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (APIServer pid=963) File "uvloop/loop.pyx", line 1518, in uvloop.loop.Loop.run_until_complete (APIServer pid=963) File "/usr/local/python3.11.13/lib/python3.11/site-packages/uvloop/__init__.py", line 48, in wrapper (APIServer pid=963) return await main (APIServer pid=963) ^^^^^^^^^^ (APIServer pid=963) File "/vllm-workspace/vllm/vllm/entrypoints/openai/api_server.py", line 1884, in run_server (APIServer pid=963) await run_server_worker(listen_address, sock, args, **uvicorn_kwargs) (APIServer pid=963) File "/vllm-workspace/vllm/vllm/entrypoints/openai/api_server.py", line 1902, in run_server_worker (APIServer pid=963) async with build_async_engine_client( (APIServer pid=963) File "/usr/local/python3.11.13/lib/python3.11/contextlib.py", line 210, in __aenter__ (APIServer pid=963) return await anext(self.gen) (APIServer pid=963) ^^^^^^^^^^^^^^^^^^^^^ (APIServer pid=963) File "/vllm-workspace/vllm/vllm/entrypoints/openai/api_server.py", line 180, in build_async_engine_client (APIServer pid=963) async with build_async_engine_client_from_engine_args( (APIServer pid=963) File "/usr/local/python3.11.13/lib/python3.11/contextlib.py", line 210, in __aenter__ (APIServer pid=963) return await anext(self.gen) (APIServer pid=963) ^^^^^^^^^^^^^^^^^^^^^ (APIServer pid=963) File "/vllm-workspace/vllm/vllm/entrypoints/openai/api_server.py", line 225, in build_async_engine_client_from_engine_args (APIServer pid=963) async_llm = AsyncLLM.from_vllm_config( (APIServer pid=963) ^^^^^^^^^^^^^^^^^^^^^^^^^^ (APIServer pid=963) File "/vllm-workspace/vllm/vllm/utils/__init__.py", line 1572, in inner (APIServer pid=963) return fn(*args, **kwargs) (APIServer pid=963) ^^^^^^^^^^^^^^^^^^^ (APIServer pid=963) File "/vllm-workspace/vllm/vllm/v1/engine/async_llm.py", line 207, in from_vllm_config (APIServer pid=963) return cls( (APIServer pid=963) ^^^^ (APIServer pid=963) File "/vllm-workspace/vllm/vllm/v1/engine/async_llm.py", line 134, in __init__ (APIServer pid=963) self.engine_core = EngineCoreClient.make_async_mp_client( (APIServer pid=963) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (APIServer pid=963) File "/vllm-workspace/vllm/vllm/v1/engine/core_client.py", line 102, in make_async_mp_client (APIServer pid=963) return AsyncMPClient(*client_args) (APIServer pid=963) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ (APIServer pid=963) File "/vllm-workspace/vllm/vllm/v1/engine/core_client.py", line 769, in __init__ (APIServer pid=963) super().__init__( (APIServer pid=963) File "/vllm-workspace/vllm/vllm/v1/engine/core_client.py", line 448, in __init__ (APIServer pid=963) with launch_core_engines(vllm_config, executor_class, (APIServer pid=963) File "/usr/local/python3.11.13/lib/python3.11/contextlib.py", line 144, in __exit__ (APIServer pid=963) next(self.gen) (APIServer pid=963) File "/vllm-workspace/vllm/vllm/v1/engine/utils.py", line 732, in launch_core_engines (APIServer pid=963) wait_for_engine_startup( (APIServer pid=963) File "/vllm-workspace/vllm/vllm/v1/engine/utils.py", line 785, in wait_for_engine_startup (APIServer pid=963) raise RuntimeError("Engine core initialization failed. " (APIServer pid=963) RuntimeError: Engine core initialization failed. See root cause above. Failed core proc(s): {} (APIServer pid=963) [ERROR] 2026-01-07-06:26:34 (PID:963, Device:-1, RankID:-1) ERR99999 UNKNOWN applicaiton exception异常时的npu信息
想请问下报错是参数问题还是硬件不支持，如何正确在一张卡上运行多个模型服务？