提问： 300I-Duo上支持vllm_npu么？支持哪个版本呢？如何解决下面的报错呢？
------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
设备： 300I-Duo
镜像：官方镜像 mindie:1.0.0-300I-Duo-py311-openeuler24.03-lts
容器创建命令: docker run -it -d --net=host --shm-size=32g --privileged --device=/dev/hisi_hdc --device=/dev/devmm_svm --name mindie -v /usr/local/Ascend/driver:/usr/local/Ascend/driver:ro -v /usr/local/sbin:/usr/local/sbin:ro -v /data:/data swr.cn-south-1.myhuaweicloud.com/ascendhub/mindie:1.0.0-300I-Duo-py311-openeuler24.03-lts bash
------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
下载模型文件后， 修改模型config.py文件中dtype为float16， 在 /usr/local/Ascend/atb-models 目录下单卡执行 bash examples/models/qwen/run_pa.sh -m /data/.llmModelCache/modelscope/hub/Qwen/Qwen2___5-7B-Instruct --trust_remote_code true 有正常输出(transformers =4.43.1 和4.46.1均可)。
------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
参照官方文档安装vllm_npu-0.3.3 (https://www.hiascend.com/document/detail/zh/mindie/100/mindieservice/Thirdpartyservitization/mindie_openthird_0010.html)后，修改examples/test_offline.sh中的模型路径为上述qwen2.5-7B模型路径, 修改examples/test_offline.py中tensor_parallel_size=1。
执行bash test_offline.sh,报错（transformers =4.43.1 和4.46.1均报错）。
执行脚本后，终端输出如下：
[root@localhost examples]# bash test_offline.sh
/usr/local/lib64/python3.11/site-packages/torch_npu/contrib/transfer_to_npu.py:288: ImportWarning:
*************************************************************************************************************
The torch.Tensor.cuda and torch.nn.Module.cuda are replaced with torch.Tensor.npu and torch.nn.Module.npu now..
The torch.cuda.DoubleTensor is replaced with torch.npu.FloatTensor cause the double type is not supported now..
The backend in torch.distributed.init_process_group set to hccl now..
The torch.cuda.* and torch.cuda.amp.* are replaced with torch.npu.* and torch.npu.amp.* now..
The device parameters have been replaced with npu in the function below:
torch.logspace, torch.randint, torch.hann_window, torch.rand, torch.full_like, torch.ones_like, torch.rand_like, torch.randperm, torch.arange, torch.frombuffer, torch.normal, torch._empty_per_channel_affine_quantized, torch.empty_strided, torch.empty_like, torch.scalar_tensor, torch.tril_indices, torch.bartlett_window, torch.ones, torch.sparse_coo_tensor, torch.randn, torch.kaiser_window, torch.tensor, torch.triu_indices, torch.as_tensor, torch.zeros, torch.randint_like, torch.full, torch.eye, torch._sparse_csr_tensor_unsafe, torch.empty, torch._sparse_coo_tensor_unsafe, torch.blackman_window, torch.zeros_like, torch.range, torch.sparse_csr_tensor, torch.randn_like, torch.from_file, torch._cudnn_init_dropout_state, torch._empty_affine_quantized, torch.linspace, torch.hamming_window, torch.empty_quantized, torch._pin_memory, torch.autocast, torch.load, torch.Generator, torch.set_default_device, torch.Tensor.new_empty, torch.Tensor.new_empty_strided, torch.Tensor.new_full, torch.Tensor.new_ones, torch.Tensor.new_tensor, torch.Tensor.new_zeros, torch.Tensor.to, torch.nn.Module.to, torch.nn.Module.to_empty
*************************************************************************************************************
warnings.warn(msg, ImportWarning)
/usr/local/lib64/python3.11/site-packages/torch_npu/contrib/transfer_to_npu.py:247: RuntimeWarning: torch.jit.script and torch.jit.script_method will be disabled by transfer_to_npu, which currently does not support them, if you need to enable them, please do not use transfer_to_npu.
warnings.warn(msg, RuntimeWarning)
/usr/local/lib64/python3.11/site-packages/torch_npu/dynamo/torchair/__init__.py:3: DeprecationWarning: pkg_resources is deprecated as an API. See https://setuptools.pypa.io/en/latest/pkg_resources.html
import pkg_resources
/usr/lib/python3.11/site-packages/pkg_resources/__init__.py:2871: DeprecationWarning: Deprecated call to `pkg_resources.declare_namespace('google')`.
Implementing implicit namespace packages (as specified in PEP 420) is preferred to `pkg_resources.declare_namespace`. See https://setuptools.pypa.io/en/latest/references/keywords.html#keyword-namespace-packages
declare_namespace(pkg)
/usr/lib/python3.11/site-packages/pkg_resources/__init__.py:2871: DeprecationWarning: Deprecated call to `pkg_resources.declare_namespace('zope')`.
Implementing implicit namespace packages (as specified in PEP 420) is preferred to `pkg_resources.declare_namespace`. See https://setuptools.pypa.io/en/latest/references/keywords.html#keyword-namespace-packages
declare_namespace(pkg)
INFO 02-20 18:13:10 llm_engine.py:87] Initializing an LLM engine with config: model='/data/.llmModelCache/modelscope/hub/Qwen/Qwen2___5-7B-Instruct', tokenizer='/data/.llmModelCache/modelscope/hub/Qwen/Qwen2___5-7B-Instruct', tokenizer_mode=auto, revision=None, tokenizer_revision=None, trust_remote_code=True, dtype=torch.float16, max_seq_len=4096, download_dir=None, load_format=auto, tensor_parallel_size=1, disable_custom_all_reduce=False, quantization=None, enforce_eager=True, kv_cache_dtype=auto, device_config=npu, seed=0)
[WARN]operator(),compiler_depend.ts:661:Feature is not supportted and the possible cause is that driver and firmware packages do not match.
[2025-02-20 18:13:18,971] [92233] [281471198513440] [llm] [INFO] [cpu_binding.py-212] : rank_id: 0, device_id: 0, numa_id: 0, shard_devices: [0, 1, 2, 3], cpus: [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20, 21, 22, 23, 24, 25, 26, 27, 28, 29, 30, 31]
[2025-02-20 18:13:18,973] [92233] [281471198513440] [llm] [INFO] [cpu_binding.py-238] : process 92233, new_affinity is [0, 1, 2, 3, 4, 5, 6, 7], cpu count 8
[2025-02-20 18:13:19,406] [92233] [281471198513440] [llm] [INFO] [logging.py-331] : model_runner.quantize: None, model_runner.kv_quant_type: None, model_runner.fa_quant_type: None, model_runner.dtype: torch.float16
[2025-02-20 18:13:19,406] [92233] [281471198513440] [llm] [INFO] [dist.py-77] : initialize_distributed has been Set
[2025-02-20 18:13:19,407] [92233] [281471198513440] [llm] [INFO] [logging.py-331] : init tokenizer done: Qwen2TokenizerFast(name_or_path='/data/.llmModelCache/modelscope/hub/Qwen/Qwen2___5-7B-Instruct', vocab_size=151643, model_max_length=131072, is_fast=True, padding_side='left', truncation_side='right', special_tokens={'eos_token': '<|im_end|>', 'pad_token': '<|endoftext|>', 'additional_special_tokens': ['<|im_start|>', '<|im_end|>', '<|object_ref_start|>', '<|object_ref_end|>', '<|box_start|>', '<|box_end|>', '<|quad_start|>', '<|quad_end|>', '<|vision_start|>', '<|vision_end|>', '<|vision_pad|>', '<|image_pad|>', '<|video_pad|>']}, clean_up_tokenization_spaces=False), added_tokens_decoder={
151643: AddedToken("<|endoftext|>", rstrip=False, lstrip=False, single_word=False, normalized=False, special=True),
151644: AddedToken("<|im_start|>", rstrip=False, lstrip=False, single_word=False, normalized=False, special=True),
151645: AddedToken("<|im_end|>", rstrip=False, lstrip=False, single_word=False, normalized=False, special=True),
151646: AddedToken("<|object_ref_start|>", rstrip=False, lstrip=False, single_word=False, normalized=False, special=True),
151647: AddedToken("<|object_ref_end|>", rstrip=False, lstrip=False, single_word=False, normalized=False, special=True),
151648: AddedToken("<|box_start|>", rstrip=False, lstrip=False, single_word=False, normalized=False, special=True),
151649: AddedToken("<|box_end|>", rstrip=False, lstrip=False, single_word=False, normalized=False, special=True),
151650: AddedToken("<|quad_start|>", rstrip=False, lstrip=False, single_word=False, normalized=False, special=True),
151651: AddedToken("<|quad_end|>", rstrip=False, lstrip=False, single_word=False, normalized=False, special=True),
151652: AddedToken("<|vision_start|>", rstrip=False, lstrip=False, single_word=False, normalized=False, special=True),
151653: AddedToken("<|vision_end|>", rstrip=False, lstrip=False, single_word=False, normalized=False, special=True),
151654: AddedToken("<|vision_pad|>", rstrip=False, lstrip=False, single_word=False, normalized=False, special=True),
151655: AddedToken("<|image_pad|>", rstrip=False, lstrip=False, single_word=False, normalized=False, special=True),
151656: AddedToken("<|video_pad|>", rstrip=False, lstrip=False, single_word=False, normalized=False, special=True),
151657: AddedToken("<tool_call>", rstrip=False, lstrip=False, single_word=False, normalized=False, special=False),
151658: AddedToken("</tool_call>", rstrip=False, lstrip=False, single_word=False, normalized=False, special=False),
151659: AddedToken("<|fim_prefix|>", rstrip=False, lstrip=False, single_word=False, normalized=False, special=False),
151660: AddedToken("<|fim_middle|>", rstrip=False, lstrip=False, single_word=False, normalized=False, special=False),
151661: AddedToken("<|fim_suffix|>", rstrip=False, lstrip=False, single_word=False, normalized=False, special=False),
151662: AddedToken("<|fim_pad|>", rstrip=False, lstrip=False, single_word=False, normalized=False, special=False),
151663: AddedToken("<|repo_name|>", rstrip=False, lstrip=False, single_word=False, normalized=False, special=False),
151664: AddedToken("<|file_sep|>", rstrip=False, lstrip=False, single_word=False, normalized=False, special=False),
}
/usr/local/lib64/python3.11/site-packages/torch/distributed/distributed_c10d.py:602: UserWarning: You are using a Backend <class 'atb_llm.utils.dist.FakeGroup'> as a ProcessGroup. This usage is deprecated since PyTorch 2.0. Please use a public API of PyTorch Distributed instead.
warnings.warn(
[2025-02-20 18:13:19,411] [92233] [281471198513440] [llm] [INFO] [logging.py-331] : NPUSocInfo(soc_name='', soc_version=202, need_nz=True, matmul_nd_nz=False)
[2025-02-20 18:13:19,487] [92233] [281471198513440] [llm] [INFO] [flash_causal_qwen2.py-115] : >>>> qwen_QwenDecoderModel is called.
[2025-02-20 18:13:27,255] [92233] [281471198513440] [llm] [INFO] [logging.py-331] : model:
FlashQwen2ForCausalLM(
(rotary_embedding): PositionRotaryEmbedding()
(attn_mask): AttentionMask()
(transformer): FlashQwenModel(
(wte): TensorParallelEmbedding()
(h): ModuleList(
(0-27): 28 x FlashQwenLayer(
(attn): FlashQwenAttention(
(rotary_emb): PositionRotaryEmbedding()
(c_attn): TensorParallelColumnLinear(
(linear): FastLinear()
)
(c_proj): TensorParallelRowLinear(
(linear): FastLinear()
)
)
(mlp): QwenMLP(
(act): SiLU()
(w2_w1): TensorParallelColumnLinear(
(linear): FastLinear()
)
(c_proj): TensorParallelRowLinear(
(linear): FastLinear()
)
)
(ln_1): QwenRMSNorm()
(ln_2): QwenRMSNorm()
)
)
(ln_f): QwenRMSNorm()
)
(lm_head): TensorParallelHead(
(linear): FastLinear()
)
)
[2025-02-20 18:13:27,317] [92233] [281471198513440] [llm] [INFO] [logging.py-331] : trans to 29
[2025-02-20 18:13:27,317] [92233] [281471198513440] [llm] [INFO] [logging.py-331] : trans to 29
[2025-02-20 18:13:27,319] [92233] [281471198513440] [llm] [INFO] [logging.py-331] : trans to 29
[2025-02-20 18:13:27,319] [92233] [281471198513440] [llm] [INFO] [logging.py-331] : trans to 29
[2025-02-20 18:13:27,320] [92233] [281471198513440] [llm] [INFO] [logging.py-331] : trans to 29
[2025-02-20 18:13:27,320] [92233] [281471198513440] [llm] [INFO] [logging.py-331] : trans to 29
[2025-02-20 18:13:27,321] [92233] [281471198513440] [llm] [INFO] [logging.py-331] : trans to 29
[2025-02-20 18:13:27,322] [92233] [281471198513440] [llm] [INFO] [logging.py-331] : trans to 29
[2025-02-20 18:13:27,322] [92233] [281471198513440] [llm] [INFO] [logging.py-331] : trans to 29
[2025-02-20 18:13:27,323] [92233] [281471198513440] [llm] [INFO] [logging.py-331] : trans to 29
[2025-02-20 18:13:27,323] [92233] [281471198513440] [llm] [INFO] [logging.py-331] : trans to 29
[2025-02-20 18:13:27,324] [92233] [281471198513440] [llm] [INFO] [logging.py-331] : trans to 29
[2025-02-20 18:13:27,324] [92233] [281471198513440] [llm] [INFO] [logging.py-331] : trans to 29
[2025-02-20 18:13:27,324] [92233] [281471198513440] [llm] [INFO] [logging.py-331] : trans to 29
[2025-02-20 18:13:27,325] [92233] [281471198513440] [llm] [INFO] [logging.py-331] : trans to 29
[2025-02-20 18:13:27,325] [92233] [281471198513440] [llm] [INFO] [logging.py-331] : trans to 29
[2025-02-20 18:13:27,326] [92233] [281471198513440] [llm] [INFO] [logging.py-331] : trans to 29
[2025-02-20 18:13:27,326] [92233] [281471198513440] [llm] [INFO] [logging.py-331] : trans to 29
[2025-02-20 18:13:27,326] [92233] [281471198513440] [llm] [INFO] [logging.py-331] : trans to 29
[2025-02-20 18:13:27,327] [92233] [281471198513440] [llm] [INFO] [logging.py-331] : trans to 29
[2025-02-20 18:13:27,327] [92233] [281471198513440] [llm] [INFO] [logging.py-331] : trans to 29
[2025-02-20 18:13:27,328] [92233] [281471198513440] [llm] [INFO] [logging.py-331] : trans to 29
[2025-02-20 18:13:27,328] [92233] [281471198513440] [llm] [INFO] [logging.py-331] : trans to 29
[2025-02-20 18:13:27,329] [92233] [281471198513440] [llm] [INFO] [logging.py-331] : trans to 29
[2025-02-20 18:13:27,329] [92233] [281471198513440] [llm] [INFO] [logging.py-331] : trans to 29
[2025-02-20 18:13:27,329] [92233] [281471198513440] [llm] [INFO] [logging.py-331] : trans to 29
[2025-02-20 18:13:27,330] [92233] [281471198513440] [llm] [INFO] [logging.py-331] : trans to 29
[2025-02-20 18:13:27,330] [92233] [281471198513440] [llm] [INFO] [logging.py-331] : trans to 29
[2025-02-20 18:13:27,331] [92233] [281471198513440] [llm] [INFO] [logging.py-331] : trans to 29
[2025-02-20 18:13:27,331] [92233] [281471198513440] [llm] [INFO] [logging.py-331] : trans to 29
[2025-02-20 18:13:27,331] [92233] [281471198513440] [llm] [INFO] [logging.py-331] : trans to 29
[2025-02-20 18:13:27,332] [92233] [281471198513440] [llm] [INFO] [logging.py-331] : trans to 29
[2025-02-20 18:13:27,332] [92233] [281471198513440] [llm] [INFO] [logging.py-331] : trans to 29
[2025-02-20 18:13:27,333] [92233] [281471198513440] [llm] [INFO] [logging.py-331] : trans to 29
[2025-02-20 18:13:27,333] [92233] [281471198513440] [llm] [INFO] [logging.py-331] : trans to 29
[2025-02-20 18:13:27,334] [92233] [281471198513440] [llm] [INFO] [logging.py-331] : trans to 29
[2025-02-20 18:13:27,334] [92233] [281471198513440] [llm] [INFO] [logging.py-331] : trans to 29
[2025-02-20 18:13:27,335] [92233] [281471198513440] [llm] [INFO] [logging.py-331] : trans to 29
[2025-02-20 18:13:27,335] [92233] [281471198513440] [llm] [INFO] [logging.py-331] : trans to 29
[2025-02-20 18:13:27,335] [92233] [281471198513440] [llm] [INFO] [logging.py-331] : trans to 29
[2025-02-20 18:13:27,336] [92233] [281471198513440] [llm] [INFO] [logging.py-331] : trans to 29
[2025-02-20 18:13:27,336] [92233] [281471198513440] [llm] [INFO] [logging.py-331] : trans to 29
[2025-02-20 18:13:27,337] [92233] [281471198513440] [llm] [INFO] [logging.py-331] : trans to 29
[2025-02-20 18:13:27,337] [92233] [281471198513440] [llm] [INFO] [logging.py-331] : trans to 29
[2025-02-20 18:13:27,338] [92233] [281471198513440] [llm] [INFO] [logging.py-331] : trans to 29
[2025-02-20 18:13:27,338] [92233] [281471198513440] [llm] [INFO] [logging.py-331] : trans to 29
[2025-02-20 18:13:27,338] [92233] [281471198513440] [llm] [INFO] [logging.py-331] : trans to 29
[2025-02-20 18:13:27,339] [92233] [281471198513440] [llm] [INFO] [logging.py-331] : trans to 29
[2025-02-20 18:13:27,339] [92233] [281471198513440] [llm] [INFO] [logging.py-331] : trans to 29
[2025-02-20 18:13:27,339] [92233] [281471198513440] [llm] [INFO] [logging.py-331] : trans to 29
[2025-02-20 18:13:27,340] [92233] [281471198513440] [llm] [INFO] [logging.py-331] : trans to 29
[2025-02-20 18:13:27,340] [92233] [281471198513440] [llm] [INFO] [logging.py-331] : trans to 29
[2025-02-20 18:13:27,341] [92233] [281471198513440] [llm] [INFO] [logging.py-331] : trans to 29
[2025-02-20 18:13:27,341] [92233] [281471198513440] [llm] [INFO] [logging.py-331] : trans to 29
[2025-02-20 18:13:27,342] [92233] [281471198513440] [llm] [INFO] [logging.py-331] : trans to 29
[2025-02-20 18:13:27,342] [92233] [281471198513440] [llm] [INFO] [logging.py-331] : trans to 29
[2025-02-20 18:13:27,343] [92233] [281471198513440] [llm] [INFO] [logging.py-331] : trans to 29
[2025-02-20 18:13:27,343] [92233] [281471198513440] [llm] [INFO] [logging.py-331] : trans to 29
[2025-02-20 18:13:27,343] [92233] [281471198513440] [llm] [INFO] [logging.py-331] : trans to 29
[2025-02-20 18:13:27,344] [92233] [281471198513440] [llm] [INFO] [logging.py-331] : trans to 29
[2025-02-20 18:13:27,344] [92233] [281471198513440] [llm] [INFO] [logging.py-331] : trans to 29
[2025-02-20 18:13:27,345] [92233] [281471198513440] [llm] [INFO] [logging.py-331] : trans to 29
[2025-02-20 18:13:27,345] [92233] [281471198513440] [llm] [INFO] [logging.py-331] : trans to 29
[2025-02-20 18:13:27,346] [92233] [281471198513440] [llm] [INFO] [logging.py-331] : trans to 29
[2025-02-20 18:13:27,346] [92233] [281471198513440] [llm] [INFO] [logging.py-331] : trans to 29
[2025-02-20 18:13:27,347] [92233] [281471198513440] [llm] [INFO] [logging.py-331] : trans to 29
[2025-02-20 18:13:27,347] [92233] [281471198513440] [llm] [INFO] [logging.py-331] : trans to 29
[2025-02-20 18:13:27,347] [92233] [281471198513440] [llm] [INFO] [logging.py-331] : trans to 29
[2025-02-20 18:13:27,348] [92233] [281471198513440] [llm] [INFO] [logging.py-331] : trans to 29
[2025-02-20 18:13:27,348] [92233] [281471198513440] [llm] [INFO] [logging.py-331] : trans to 29
[2025-02-20 18:13:27,349] [92233] [281471198513440] [llm] [INFO] [logging.py-331] : trans to 29
[2025-02-20 18:13:27,349] [92233] [281471198513440] [llm] [INFO] [logging.py-331] : trans to 29
[2025-02-20 18:13:27,349] [92233] [281471198513440] [llm] [INFO] [logging.py-331] : trans to 29
[2025-02-20 18:13:27,350] [92233] [281471198513440] [llm] [INFO] [logging.py-331] : trans to 29
[2025-02-20 18:13:27,350] [92233] [281471198513440] [llm] [INFO] [logging.py-331] : trans to 29
[2025-02-20 18:13:27,350] [92233] [281471198513440] [llm] [INFO] [logging.py-331] : trans to 29
[2025-02-20 18:13:27,351] [92233] [281471198513440] [llm] [INFO] [logging.py-331] : trans to 29
[2025-02-20 18:13:27,351] [92233] [281471198513440] [llm] [INFO] [logging.py-331] : trans to 29
[2025-02-20 18:13:27,352] [92233] [281471198513440] [llm] [INFO] [logging.py-331] : trans to 29
[2025-02-20 18:13:27,352] [92233] [281471198513440] [llm] [INFO] [logging.py-331] : trans to 29
[2025-02-20 18:13:27,352] [92233] [281471198513440] [llm] [INFO] [logging.py-331] : trans to 29
[2025-02-20 18:13:27,353] [92233] [281471198513440] [llm] [INFO] [logging.py-331] : trans to 29
[2025-02-20 18:13:27,353] [92233] [281471198513440] [llm] [INFO] [logging.py-331] : trans to 29
[2025-02-20 18:13:27,353] [92233] [281471198513440] [llm] [INFO] [logging.py-331] : trans to 29
[2025-02-20 18:13:27,354] [92233] [281471198513440] [llm] [INFO] [logging.py-331] : trans to 29
[2025-02-20 18:13:27,354] [92233] [281471198513440] [llm] [INFO] [logging.py-331] : trans to 29
[2025-02-20 18:13:27,355] [92233] [281471198513440] [llm] [INFO] [logging.py-331] : trans to 29
[2025-02-20 18:13:27,355] [92233] [281471198513440] [llm] [INFO] [logging.py-331] : trans to 29
[2025-02-20 18:13:27,356] [92233] [281471198513440] [llm] [INFO] [logging.py-331] : trans to 29
[2025-02-20 18:13:27,356] [92233] [281471198513440] [llm] [INFO] [logging.py-331] : trans to 29
[2025-02-20 18:13:27,357] [92233] [281471198513440] [llm] [INFO] [logging.py-331] : trans to 29
[2025-02-20 18:13:27,357] [92233] [281471198513440] [llm] [INFO] [logging.py-331] : trans to 29
[2025-02-20 18:13:27,358] [92233] [281471198513440] [llm] [INFO] [logging.py-331] : trans to 29
[2025-02-20 18:13:27,358] [92233] [281471198513440] [llm] [INFO] [logging.py-331] : trans to 29
[2025-02-20 18:13:27,358] [92233] [281471198513440] [llm] [INFO] [logging.py-331] : trans to 29
[2025-02-20 18:13:27,359] [92233] [281471198513440] [llm] [INFO] [logging.py-331] : trans to 29
[2025-02-20 18:13:27,359] [92233] [281471198513440] [llm] [INFO] [logging.py-331] : trans to 29
[2025-02-20 18:13:27,360] [92233] [281471198513440] [llm] [INFO] [logging.py-331] : trans to 29
[2025-02-20 18:13:27,360] [92233] [281471198513440] [llm] [INFO] [logging.py-331] : trans to 29
[2025-02-20 18:13:27,361] [92233] [281471198513440] [llm] [INFO] [logging.py-331] : trans to 29
[2025-02-20 18:13:27,361] [92233] [281471198513440] [llm] [INFO] [logging.py-331] : trans to 29
[2025-02-20 18:13:27,362] [92233] [281471198513440] [llm] [INFO] [logging.py-331] : trans to 29
[2025-02-20 18:13:27,362] [92233] [281471198513440] [llm] [INFO] [logging.py-331] : trans to 29
[2025-02-20 18:13:27,362] [92233] [281471198513440] [llm] [INFO] [logging.py-331] : trans to 29
[2025-02-20 18:13:27,363] [92233] [281471198513440] [llm] [INFO] [logging.py-331] : trans to 29
[2025-02-20 18:13:27,363] [92233] [281471198513440] [llm] [INFO] [logging.py-331] : trans to 29
[2025-02-20 18:13:27,363] [92233] [281471198513440] [llm] [INFO] [logging.py-331] : trans to 29
[2025-02-20 18:13:27,364] [92233] [281471198513440] [llm] [INFO] [logging.py-331] : trans to 29
[2025-02-20 18:13:27,364] [92233] [281471198513440] [llm] [INFO] [logging.py-331] : trans to 29
[2025-02-20 18:13:27,365] [92233] [281471198513440] [llm] [INFO] [logging.py-331] : trans to 29
[2025-02-20 18:13:27,365] [92233] [281471198513440] [llm] [INFO] [logging.py-331] : trans to 29
[2025-02-20 18:13:27,365] [92233] [281471198513440] [llm] [INFO] [logging.py-331] : trans to 29
[2025-02-20 18:13:27,370] [92233] [281471198513440] [llm] [INFO] [logging.py-331] : trans to 29
[2025-02-20 18:13:27,994] [92233] [281471198513440] [llm] [INFO] [logging.py-331] : <<<<<<< ori k_caches[0].shape=torch.Size([1, 128, 4, 128])
[2025-02-20 18:13:27,996] [92233] [281471198513440] [llm] [INFO] [flash_causal_qwen2.py-394] : <<<<<<<after transdata k_caches[0].shape=torch.Size([1, 128, 4, 128])
[2025-02-20 18:13:27,997] [92233] [281471198513440] [llm] [INFO] [logging.py-331] : >>>>>>id of kcache is 281470735710256 id of vcache is 281470735710832
[2025-02-20 18:13:28.392003] [error] [92233] [reshape_and_cache_operation.cpp:288] ReshapeAndCacheOperation_3_0_3 The blockSize of keyCache and valueCache should be 16-aligned
[2025-02-20 18:13:28.392055] [error] [92233] [operation_base.cpp:267] ReshapeAndCacheOperation_3_0_3 infer shape check fail, error code: 8
[2025-02-20 18:13:28.392063] [error] [92233] [graph_runner.cpp:662] AttentionRunner_3_0:0 node[3] infer shape fail, error code: 8
[2025-02-20 18:13:28.392076] [error] [92233] [graph_runner.cpp:263] AttentionRunner_3_0:0 node[3] setup fail, PreparseNodeVariantPack fail, error code:8
[2025-02-20 18:13:28.392084] [error] [92233] [graph_runner.cpp:762] Prefill_layerRunner_3:0 node[0] setup fail, error code:8
[2025-02-20 18:13:28.392090] [error] [92233] [graph_runner.cpp:270] Prefill_layerRunner_3:0 node[0] SetupNodeRunners fail, error code:8
[2025-02-20 18:13:28.392096] [error] [92233] [operation_base.cpp:462] Prefill_layer_3 runner setup fail
[2025-02-20 18:13:28.392102] [error] [92233] [operation_base.cpp:518] Prefill_layer_3 setup fail, error code: 8
Traceback (most recent call last):
File "/data/et_llm/vllm_npu/examples/test_offline.py", line 13, in <module>
llm = LLM(model=model_path,
^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.11/site-packages/vllm-0.3.3-py3.11.egg/vllm/entrypoints/llm.py", line 109, in __init__
self.llm_engine = LLMEngine.from_engine_args(engine_args)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.11/site-packages/vllm-0.3.3-py3.11.egg/vllm/engine/llm_engine.py", line 391, in from_engine_args
engine = cls(*engine_configs,
^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.11/site-packages/vllm-0.3.3-py3.11.egg/vllm/engine/llm_engine.py", line 131, in __init__
self._init_cache()
File "/usr/local/lib/python3.11/site-packages/vllm-0.3.3-py3.11.egg/vllm/engine/llm_engine.py", line 343, in _init_cache
num_blocks = self._run_workers(
^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.11/site-packages/vllm-0.3.3-py3.11.egg/vllm/engine/llm_engine.py", line 1041, in _run_workers
driver_worker_output = getattr(self.driver_worker,
^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib64/python3.11/site-packages/torch/utils/_contextlib.py", line 115, in decorate_context
return func(*args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.11/site-packages/vllm_npu-0.3.3-py3.11.egg/vllm_npu/worker/ascend_worker.py", line 104, in profile_num_available_blocks
File "/usr/local/lib64/python3.11/site-packages/torch/utils/_contextlib.py", line 115, in decorate_context
return func(*args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.11/site-packages/vllm_npu-0.3.3-py3.11.egg/vllm_npu/worker/model_runner.py", line 267, in profile_run
File "/usr/local/lib64/python3.11/site-packages/torch/utils/_contextlib.py", line 115, in decorate_context
return func(*args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.11/site-packages/vllm_npu-0.3.3-py3.11.egg/vllm_npu/worker/model_runner.py", line 232, in execute_model
File "/usr/local/lib64/python3.11/site-packages/torch/nn/modules/module.py", line 1518, in _wrapped_call_impl
return self._call_impl(*args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib64/python3.11/site-packages/torch/nn/modules/module.py", line 1527, in _call_impl
return forward_call(*args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.11/site-packages/vllm_npu-0.3.3-py3.11.egg/vllm_npu/model_executor/models/ascend/mindie_llm_wrapper.py", line 46, in forward
File "/usr/local/lib/python3.11/site-packages/mindie_llm/text_generator/adapter/generator_torch.py", line 127, in forward_tensor
logits = self.model_wrapper.forward_tensor(
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.11/site-packages/mindie_llm/modeling/model_wrapper/atb/atb_model_wrapper.py", line 116, in forward_tensor
logits = self.model_runner.forward(
^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/Ascend/atb-models/atb_llm/runner/model_runner.py", line 193, in forward
return self.model.forward(**kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/Ascend/atb-models/atb_llm/models/base/flash_causal_lm.py", line 458, in forward
logits = self.execute_ascend_operator(acl_inputs, acl_param, is_prefill)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/Ascend/atb-models/atb_llm/models/qwen2/flash_causal_qwen2.py", line 374, in execute_ascend_operator
acl_model_out = model_operation.execute(acl_inputs, acl_param)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
RuntimeError: Setup fail, enable log: export ASDOPS_LOG_LEVEL=ERROR, export ASDOPS_LOG_TO_STDOUT=1 to find the first error. For more details, see the MindIE official document
------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
python相关库版本如下：
点赞
收藏
回复
分享
举报
已解决
提问： 300I-Duo上支持vllm_npu么？支持哪个版本呢？如何解决下面的报错呢？
------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
设备： 300I-Duo
镜像：官方镜像 mindie:1.0.0-300I-Duo-py311-openeuler24.03-lts
容器创建命令: docker run -it -d --net=host --shm-size=32g --privileged --device=/dev/hisi_hdc --device=/dev/devmm_svm --name mindie -v /usr/local/Ascend/driver:/usr/local/Ascend/driver:ro -v /usr/local/sbin:/usr/local/sbin:ro -v /data:/data swr.cn-south-1.myhuaweicloud.com/ascendhub/mindie:1.0.0-300I-Duo-py311-openeuler24.03-lts bash
------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
下载模型文件后， 修改模型config.py文件中dtype为float16， 在 /usr/local/Ascend/atb-models 目录下单卡执行 bash examples/models/qwen/run_pa.sh -m /data/.llmModelCache/modelscope/hub/Qwen/Qwen2___5-7B-Instruct --trust_remote_code true 有正常输出(transformers =4.43.1 和4.46.1均可)。
------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
参照官方文档安装vllm_npu-0.3.3 (https://www.hiascend.com/document/detail/zh/mindie/100/mindieservice/Thirdpartyservitization/mindie_openthird_0010.html)后，修改examples/test_offline.sh中的模型路径为上述qwen2.5-7B模型路径, 修改examples/test_offline.py中tensor_parallel_size=1。
执行bash test_offline.sh,报错（transformers =4.43.1 和4.46.1均报错）。
执行脚本后，终端输出如下：
[root@localhost examples]# bash test_offline.sh
/usr/local/lib64/python3.11/site-packages/torch_npu/contrib/transfer_to_npu.py:288: ImportWarning:
*************************************************************************************************************
The torch.Tensor.cuda and torch.nn.Module.cuda are replaced with torch.Tensor.npu and torch.nn.Module.npu now..
The torch.cuda.DoubleTensor is replaced with torch.npu.FloatTensor cause the double type is not supported now..
The backend in torch.distributed.init_process_group set to hccl now..
The torch.cuda.* and torch.cuda.amp.* are replaced with torch.npu.* and torch.npu.amp.* now..
The device parameters have been replaced with npu in the function below:
torch.logspace, torch.randint, torch.hann_window, torch.rand, torch.full_like, torch.ones_like, torch.rand_like, torch.randperm, torch.arange, torch.frombuffer, torch.normal, torch._empty_per_channel_affine_quantized, torch.empty_strided, torch.empty_like, torch.scalar_tensor, torch.tril_indices, torch.bartlett_window, torch.ones, torch.sparse_coo_tensor, torch.randn, torch.kaiser_window, torch.tensor, torch.triu_indices, torch.as_tensor, torch.zeros, torch.randint_like, torch.full, torch.eye, torch._sparse_csr_tensor_unsafe, torch.empty, torch._sparse_coo_tensor_unsafe, torch.blackman_window, torch.zeros_like, torch.range, torch.sparse_csr_tensor, torch.randn_like, torch.from_file, torch._cudnn_init_dropout_state, torch._empty_affine_quantized, torch.linspace, torch.hamming_window, torch.empty_quantized, torch._pin_memory, torch.autocast, torch.load, torch.Generator, torch.set_default_device, torch.Tensor.new_empty, torch.Tensor.new_empty_strided, torch.Tensor.new_full, torch.Tensor.new_ones, torch.Tensor.new_tensor, torch.Tensor.new_zeros, torch.Tensor.to, torch.nn.Module.to, torch.nn.Module.to_empty
*************************************************************************************************************
warnings.warn(msg, ImportWarning)
/usr/local/lib64/python3.11/site-packages/torch_npu/contrib/transfer_to_npu.py:247: RuntimeWarning: torch.jit.script and torch.jit.script_method will be disabled by transfer_to_npu, which currently does not support them, if you need to enable them, please do not use transfer_to_npu.
warnings.warn(msg, RuntimeWarning)
/usr/local/lib64/python3.11/site-packages/torch_npu/dynamo/torchair/__init__.py:3: DeprecationWarning: pkg_resources is deprecated as an API. See https://setuptools.pypa.io/en/latest/pkg_resources.html
import pkg_resources
/usr/lib/python3.11/site-packages/pkg_resources/__init__.py:2871: DeprecationWarning: Deprecated call to `pkg_resources.declare_namespace('google')`.
Implementing implicit namespace packages (as specified in PEP 420) is preferred to `pkg_resources.declare_namespace`. See https://setuptools.pypa.io/en/latest/references/keywords.html#keyword-namespace-packages
declare_namespace(pkg)
/usr/lib/python3.11/site-packages/pkg_resources/__init__.py:2871: DeprecationWarning: Deprecated call to `pkg_resources.declare_namespace('zope')`.
Implementing implicit namespace packages (as specified in PEP 420) is preferred to `pkg_resources.declare_namespace`. See https://setuptools.pypa.io/en/latest/references/keywords.html#keyword-namespace-packages
declare_namespace(pkg)
INFO 02-20 18:13:10 llm_engine.py:87] Initializing an LLM engine with config: model='/data/.llmModelCache/modelscope/hub/Qwen/Qwen2___5-7B-Instruct', tokenizer='/data/.llmModelCache/modelscope/hub/Qwen/Qwen2___5-7B-Instruct', tokenizer_mode=auto, revision=None, tokenizer_revision=None, trust_remote_code=True, dtype=torch.float16, max_seq_len=4096, download_dir=None, load_format=auto, tensor_parallel_size=1, disable_custom_all_reduce=False, quantization=None, enforce_eager=True, kv_cache_dtype=auto, device_config=npu, seed=0)
[WARN]operator(),compiler_depend.ts:661:Feature is not supportted and the possible cause is that driver and firmware packages do not match.
[2025-02-20 18:13:18,971] [92233] [281471198513440] [llm] [INFO] [cpu_binding.py-212] : rank_id: 0, device_id: 0, numa_id: 0, shard_devices: [0, 1, 2, 3], cpus: [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20, 21, 22, 23, 24, 25, 26, 27, 28, 29, 30, 31]
[2025-02-20 18:13:18,973] [92233] [281471198513440] [llm] [INFO] [cpu_binding.py-238] : process 92233, new_affinity is [0, 1, 2, 3, 4, 5, 6, 7], cpu count 8
[2025-02-20 18:13:19,406] [92233] [281471198513440] [llm] [INFO] [logging.py-331] : model_runner.quantize: None, model_runner.kv_quant_type: None, model_runner.fa_quant_type: None, model_runner.dtype: torch.float16
[2025-02-20 18:13:19,406] [92233] [281471198513440] [llm] [INFO] [dist.py-77] : initialize_distributed has been Set
[2025-02-20 18:13:19,407] [92233] [281471198513440] [llm] [INFO] [logging.py-331] : init tokenizer done: Qwen2TokenizerFast(name_or_path='/data/.llmModelCache/modelscope/hub/Qwen/Qwen2___5-7B-Instruct', vocab_size=151643, model_max_length=131072, is_fast=True, padding_side='left', truncation_side='right', special_tokens={'eos_token': '<|im_end|>', 'pad_token': '<|endoftext|>', 'additional_special_tokens': ['<|im_start|>', '<|im_end|>', '<|object_ref_start|>', '<|object_ref_end|>', '<|box_start|>', '<|box_end|>', '<|quad_start|>', '<|quad_end|>', '<|vision_start|>', '<|vision_end|>', '<|vision_pad|>', '<|image_pad|>', '<|video_pad|>']}, clean_up_tokenization_spaces=False), added_tokens_decoder={
151643: AddedToken("<|endoftext|>", rstrip=False, lstrip=False, single_word=False, normalized=False, special=True),
151644: AddedToken("<|im_start|>", rstrip=False, lstrip=False, single_word=False, normalized=False, special=True),
151645: AddedToken("<|im_end|>", rstrip=False, lstrip=False, single_word=False, normalized=False, special=True),
151646: AddedToken("<|object_ref_start|>", rstrip=False, lstrip=False, single_word=False, normalized=False, special=True),
151647: AddedToken("<|object_ref_end|>", rstrip=False, lstrip=False, single_word=False, normalized=False, special=True),
151648: AddedToken("<|box_start|>", rstrip=False, lstrip=False, single_word=False, normalized=False, special=True),
151649: AddedToken("<|box_end|>", rstrip=False, lstrip=False, single_word=False, normalized=False, special=True),
151650: AddedToken("<|quad_start|>", rstrip=False, lstrip=False, single_word=False, normalized=False, special=True),
151651: AddedToken("<|quad_end|>", rstrip=False, lstrip=False, single_word=False, normalized=False, special=True),
151652: AddedToken("<|vision_start|>", rstrip=False, lstrip=False, single_word=False, normalized=False, special=True),
151653: AddedToken("<|vision_end|>", rstrip=False, lstrip=False, single_word=False, normalized=False, special=True),
151654: AddedToken("<|vision_pad|>", rstrip=False, lstrip=False, single_word=False, normalized=False, special=True),
151655: AddedToken("<|image_pad|>", rstrip=False, lstrip=False, single_word=False, normalized=False, special=True),
151656: AddedToken("<|video_pad|>", rstrip=False, lstrip=False, single_word=False, normalized=False, special=True),
151657: AddedToken("<tool_call>", rstrip=False, lstrip=False, single_word=False, normalized=False, special=False),
151658: AddedToken("</tool_call>", rstrip=False, lstrip=False, single_word=False, normalized=False, special=False),
151659: AddedToken("<|fim_prefix|>", rstrip=False, lstrip=False, single_word=False, normalized=False, special=False),
151660: AddedToken("<|fim_middle|>", rstrip=False, lstrip=False, single_word=False, normalized=False, special=False),
151661: AddedToken("<|fim_suffix|>", rstrip=False, lstrip=False, single_word=False, normalized=False, special=False),
151662: AddedToken("<|fim_pad|>", rstrip=False, lstrip=False, single_word=False, normalized=False, special=False),
151663: AddedToken("<|repo_name|>", rstrip=False, lstrip=False, single_word=False, normalized=False, special=False),
151664: AddedToken("<|file_sep|>", rstrip=False, lstrip=False, single_word=False, normalized=False, special=False),
}
/usr/local/lib64/python3.11/site-packages/torch/distributed/distributed_c10d.py:602: UserWarning: You are using a Backend <class 'atb_llm.utils.dist.FakeGroup'> as a ProcessGroup. This usage is deprecated since PyTorch 2.0. Please use a public API of PyTorch Distributed instead.
warnings.warn(
[2025-02-20 18:13:19,411] [92233] [281471198513440] [llm] [INFO] [logging.py-331] : NPUSocInfo(soc_name='', soc_version=202, need_nz=True, matmul_nd_nz=False)
[2025-02-20 18:13:19,487] [92233] [281471198513440] [llm] [INFO] [flash_causal_qwen2.py-115] : >>>> qwen_QwenDecoderModel is called.
[2025-02-20 18:13:27,255] [92233] [281471198513440] [llm] [INFO] [logging.py-331] : model:
FlashQwen2ForCausalLM(
(rotary_embedding): PositionRotaryEmbedding()
(attn_mask): AttentionMask()
(transformer): FlashQwenModel(
(wte): TensorParallelEmbedding()
(h): ModuleList(
(0-27): 28 x FlashQwenLayer(
(attn): FlashQwenAttention(
(rotary_emb): PositionRotaryEmbedding()
(c_attn): TensorParallelColumnLinear(
(linear): FastLinear()
)
(c_proj): TensorParallelRowLinear(
(linear): FastLinear()
)
)
(mlp): QwenMLP(
(act): SiLU()
(w2_w1): TensorParallelColumnLinear(
(linear): FastLinear()
)
(c_proj): TensorParallelRowLinear(
(linear): FastLinear()
)
)
(ln_1): QwenRMSNorm()
(ln_2): QwenRMSNorm()
)
)
(ln_f): QwenRMSNorm()
)
(lm_head): TensorParallelHead(
(linear): FastLinear()
)
)
[2025-02-20 18:13:27,317] [92233] [281471198513440] [llm] [INFO] [logging.py-331] : trans to 29
[2025-02-20 18:13:27,317] [92233] [281471198513440] [llm] [INFO] [logging.py-331] : trans to 29
[2025-02-20 18:13:27,319] [92233] [281471198513440] [llm] [INFO] [logging.py-331] : trans to 29
[2025-02-20 18:13:27,319] [92233] [281471198513440] [llm] [INFO] [logging.py-331] : trans to 29
[2025-02-20 18:13:27,320] [92233] [281471198513440] [llm] [INFO] [logging.py-331] : trans to 29
[2025-02-20 18:13:27,320] [92233] [281471198513440] [llm] [INFO] [logging.py-331] : trans to 29
[2025-02-20 18:13:27,321] [92233] [281471198513440] [llm] [INFO] [logging.py-331] : trans to 29
[2025-02-20 18:13:27,322] [92233] [281471198513440] [llm] [INFO] [logging.py-331] : trans to 29
[2025-02-20 18:13:27,322] [92233] [281471198513440] [llm] [INFO] [logging.py-331] : trans to 29
[2025-02-20 18:13:27,323] [92233] [281471198513440] [llm] [INFO] [logging.py-331] : trans to 29
[2025-02-20 18:13:27,323] [92233] [281471198513440] [llm] [INFO] [logging.py-331] : trans to 29
[2025-02-20 18:13:27,324] [92233] [281471198513440] [llm] [INFO] [logging.py-331] : trans to 29
[2025-02-20 18:13:27,324] [92233] [281471198513440] [llm] [INFO] [logging.py-331] : trans to 29
[2025-02-20 18:13:27,324] [92233] [281471198513440] [llm] [INFO] [logging.py-331] : trans to 29
[2025-02-20 18:13:27,325] [92233] [281471198513440] [llm] [INFO] [logging.py-331] : trans to 29
[2025-02-20 18:13:27,325] [92233] [281471198513440] [llm] [INFO] [logging.py-331] : trans to 29
[2025-02-20 18:13:27,326] [92233] [281471198513440] [llm] [INFO] [logging.py-331] : trans to 29
[2025-02-20 18:13:27,326] [92233] [281471198513440] [llm] [INFO] [logging.py-331] : trans to 29
[2025-02-20 18:13:27,326] [92233] [281471198513440] [llm] [INFO] [logging.py-331] : trans to 29
[2025-02-20 18:13:27,327] [92233] [281471198513440] [llm] [INFO] [logging.py-331] : trans to 29
[2025-02-20 18:13:27,327] [92233] [281471198513440] [llm] [INFO] [logging.py-331] : trans to 29
[2025-02-20 18:13:27,328] [92233] [281471198513440] [llm] [INFO] [logging.py-331] : trans to 29
[2025-02-20 18:13:27,328] [92233] [281471198513440] [llm] [INFO] [logging.py-331] : trans to 29
[2025-02-20 18:13:27,329] [92233] [281471198513440] [llm] [INFO] [logging.py-331] : trans to 29
[2025-02-20 18:13:27,329] [92233] [281471198513440] [llm] [INFO] [logging.py-331] : trans to 29
[2025-02-20 18:13:27,329] [92233] [281471198513440] [llm] [INFO] [logging.py-331] : trans to 29
[2025-02-20 18:13:27,330] [92233] [281471198513440] [llm] [INFO] [logging.py-331] : trans to 29
[2025-02-20 18:13:27,330] [92233] [281471198513440] [llm] [INFO] [logging.py-331] : trans to 29
[2025-02-20 18:13:27,331] [92233] [281471198513440] [llm] [INFO] [logging.py-331] : trans to 29
[2025-02-20 18:13:27,331] [92233] [281471198513440] [llm] [INFO] [logging.py-331] : trans to 29
[2025-02-20 18:13:27,331] [92233] [281471198513440] [llm] [INFO] [logging.py-331] : trans to 29
[2025-02-20 18:13:27,332] [92233] [281471198513440] [llm] [INFO] [logging.py-331] : trans to 29
[2025-02-20 18:13:27,332] [92233] [281471198513440] [llm] [INFO] [logging.py-331] : trans to 29
[2025-02-20 18:13:27,333] [92233] [281471198513440] [llm] [INFO] [logging.py-331] : trans to 29
[2025-02-20 18:13:27,333] [92233] [281471198513440] [llm] [INFO] [logging.py-331] : trans to 29
[2025-02-20 18:13:27,334] [92233] [281471198513440] [llm] [INFO] [logging.py-331] : trans to 29
[2025-02-20 18:13:27,334] [92233] [281471198513440] [llm] [INFO] [logging.py-331] : trans to 29
[2025-02-20 18:13:27,335] [92233] [281471198513440] [llm] [INFO] [logging.py-331] : trans to 29
[2025-02-20 18:13:27,335] [92233] [281471198513440] [llm] [INFO] [logging.py-331] : trans to 29
[2025-02-20 18:13:27,335] [92233] [281471198513440] [llm] [INFO] [logging.py-331] : trans to 29
[2025-02-20 18:13:27,336] [92233] [281471198513440] [llm] [INFO] [logging.py-331] : trans to 29
[2025-02-20 18:13:27,336] [92233] [281471198513440] [llm] [INFO] [logging.py-331] : trans to 29
[2025-02-20 18:13:27,337] [92233] [281471198513440] [llm] [INFO] [logging.py-331] : trans to 29
[2025-02-20 18:13:27,337] [92233] [281471198513440] [llm] [INFO] [logging.py-331] : trans to 29
[2025-02-20 18:13:27,338] [92233] [281471198513440] [llm] [INFO] [logging.py-331] : trans to 29
[2025-02-20 18:13:27,338] [92233] [281471198513440] [llm] [INFO] [logging.py-331] : trans to 29
[2025-02-20 18:13:27,338] [92233] [281471198513440] [llm] [INFO] [logging.py-331] : trans to 29
[2025-02-20 18:13:27,339] [92233] [281471198513440] [llm] [INFO] [logging.py-331] : trans to 29
[2025-02-20 18:13:27,339] [92233] [281471198513440] [llm] [INFO] [logging.py-331] : trans to 29
[2025-02-20 18:13:27,339] [92233] [281471198513440] [llm] [INFO] [logging.py-331] : trans to 29
[2025-02-20 18:13:27,340] [92233] [281471198513440] [llm] [INFO] [logging.py-331] : trans to 29
[2025-02-20 18:13:27,340] [92233] [281471198513440] [llm] [INFO] [logging.py-331] : trans to 29
[2025-02-20 18:13:27,341] [92233] [281471198513440] [llm] [INFO] [logging.py-331] : trans to 29
[2025-02-20 18:13:27,341] [92233] [281471198513440] [llm] [INFO] [logging.py-331] : trans to 29
[2025-02-20 18:13:27,342] [92233] [281471198513440] [llm] [INFO] [logging.py-331] : trans to 29
[2025-02-20 18:13:27,342] [92233] [281471198513440] [llm] [INFO] [logging.py-331] : trans to 29
[2025-02-20 18:13:27,343] [92233] [281471198513440] [llm] [INFO] [logging.py-331] : trans to 29
[2025-02-20 18:13:27,343] [92233] [281471198513440] [llm] [INFO] [logging.py-331] : trans to 29
[2025-02-20 18:13:27,343] [92233] [281471198513440] [llm] [INFO] [logging.py-331] : trans to 29
[2025-02-20 18:13:27,344] [92233] [281471198513440] [llm] [INFO] [logging.py-331] : trans to 29
[2025-02-20 18:13:27,344] [92233] [281471198513440] [llm] [INFO] [logging.py-331] : trans to 29
[2025-02-20 18:13:27,345] [92233] [281471198513440] [llm] [INFO] [logging.py-331] : trans to 29
[2025-02-20 18:13:27,345] [92233] [281471198513440] [llm] [INFO] [logging.py-331] : trans to 29
[2025-02-20 18:13:27,346] [92233] [281471198513440] [llm] [INFO] [logging.py-331] : trans to 29
[2025-02-20 18:13:27,346] [92233] [281471198513440] [llm] [INFO] [logging.py-331] : trans to 29
[2025-02-20 18:13:27,347] [92233] [281471198513440] [llm] [INFO] [logging.py-331] : trans to 29
[2025-02-20 18:13:27,347] [92233] [281471198513440] [llm] [INFO] [logging.py-331] : trans to 29
[2025-02-20 18:13:27,347] [92233] [281471198513440] [llm] [INFO] [logging.py-331] : trans to 29
[2025-02-20 18:13:27,348] [92233] [281471198513440] [llm] [INFO] [logging.py-331] : trans to 29
[2025-02-20 18:13:27,348] [92233] [281471198513440] [llm] [INFO] [logging.py-331] : trans to 29
[2025-02-20 18:13:27,349] [92233] [281471198513440] [llm] [INFO] [logging.py-331] : trans to 29
[2025-02-20 18:13:27,349] [92233] [281471198513440] [llm] [INFO] [logging.py-331] : trans to 29
[2025-02-20 18:13:27,349] [92233] [281471198513440] [llm] [INFO] [logging.py-331] : trans to 29
[2025-02-20 18:13:27,350] [92233] [281471198513440] [llm] [INFO] [logging.py-331] : trans to 29
[2025-02-20 18:13:27,350] [92233] [281471198513440] [llm] [INFO] [logging.py-331] : trans to 29
[2025-02-20 18:13:27,350] [92233] [281471198513440] [llm] [INFO] [logging.py-331] : trans to 29
[2025-02-20 18:13:27,351] [92233] [281471198513440] [llm] [INFO] [logging.py-331] : trans to 29
[2025-02-20 18:13:27,351] [92233] [281471198513440] [llm] [INFO] [logging.py-331] : trans to 29
[2025-02-20 18:13:27,352] [92233] [281471198513440] [llm] [INFO] [logging.py-331] : trans to 29
[2025-02-20 18:13:27,352] [92233] [281471198513440] [llm] [INFO] [logging.py-331] : trans to 29
[2025-02-20 18:13:27,352] [92233] [281471198513440] [llm] [INFO] [logging.py-331] : trans to 29
[2025-02-20 18:13:27,353] [92233] [281471198513440] [llm] [INFO] [logging.py-331] : trans to 29
[2025-02-20 18:13:27,353] [92233] [281471198513440] [llm] [INFO] [logging.py-331] : trans to 29
[2025-02-20 18:13:27,353] [92233] [281471198513440] [llm] [INFO] [logging.py-331] : trans to 29
[2025-02-20 18:13:27,354] [92233] [281471198513440] [llm] [INFO] [logging.py-331] : trans to 29
[2025-02-20 18:13:27,354] [92233] [281471198513440] [llm] [INFO] [logging.py-331] : trans to 29
[2025-02-20 18:13:27,355] [92233] [281471198513440] [llm] [INFO] [logging.py-331] : trans to 29
[2025-02-20 18:13:27,355] [92233] [281471198513440] [llm] [INFO] [logging.py-331] : trans to 29
[2025-02-20 18:13:27,356] [92233] [281471198513440] [llm] [INFO] [logging.py-331] : trans to 29
[2025-02-20 18:13:27,356] [92233] [281471198513440] [llm] [INFO] [logging.py-331] : trans to 29
[2025-02-20 18:13:27,357] [92233] [281471198513440] [llm] [INFO] [logging.py-331] : trans to 29
[2025-02-20 18:13:27,357] [92233] [281471198513440] [llm] [INFO] [logging.py-331] : trans to 29
[2025-02-20 18:13:27,358] [92233] [281471198513440] [llm] [INFO] [logging.py-331] : trans to 29
[2025-02-20 18:13:27,358] [92233] [281471198513440] [llm] [INFO] [logging.py-331] : trans to 29
[2025-02-20 18:13:27,358] [92233] [281471198513440] [llm] [INFO] [logging.py-331] : trans to 29
[2025-02-20 18:13:27,359] [92233] [281471198513440] [llm] [INFO] [logging.py-331] : trans to 29
[2025-02-20 18:13:27,359] [92233] [281471198513440] [llm] [INFO] [logging.py-331] : trans to 29
[2025-02-20 18:13:27,360] [92233] [281471198513440] [llm] [INFO] [logging.py-331] : trans to 29
[2025-02-20 18:13:27,360] [92233] [281471198513440] [llm] [INFO] [logging.py-331] : trans to 29
[2025-02-20 18:13:27,361] [92233] [281471198513440] [llm] [INFO] [logging.py-331] : trans to 29
[2025-02-20 18:13:27,361] [92233] [281471198513440] [llm] [INFO] [logging.py-331] : trans to 29
[2025-02-20 18:13:27,362] [92233] [281471198513440] [llm] [INFO] [logging.py-331] : trans to 29
[2025-02-20 18:13:27,362] [92233] [281471198513440] [llm] [INFO] [logging.py-331] : trans to 29
[2025-02-20 18:13:27,362] [92233] [281471198513440] [llm] [INFO] [logging.py-331] : trans to 29
[2025-02-20 18:13:27,363] [92233] [281471198513440] [llm] [INFO] [logging.py-331] : trans to 29
[2025-02-20 18:13:27,363] [92233] [281471198513440] [llm] [INFO] [logging.py-331] : trans to 29
[2025-02-20 18:13:27,363] [92233] [281471198513440] [llm] [INFO] [logging.py-331] : trans to 29
[2025-02-20 18:13:27,364] [92233] [281471198513440] [llm] [INFO] [logging.py-331] : trans to 29
[2025-02-20 18:13:27,364] [92233] [281471198513440] [llm] [INFO] [logging.py-331] : trans to 29
[2025-02-20 18:13:27,365] [92233] [281471198513440] [llm] [INFO] [logging.py-331] : trans to 29
[2025-02-20 18:13:27,365] [92233] [281471198513440] [llm] [INFO] [logging.py-331] : trans to 29
[2025-02-20 18:13:27,365] [92233] [281471198513440] [llm] [INFO] [logging.py-331] : trans to 29
[2025-02-20 18:13:27,370] [92233] [281471198513440] [llm] [INFO] [logging.py-331] : trans to 29
[2025-02-20 18:13:27,994] [92233] [281471198513440] [llm] [INFO] [logging.py-331] : <<<<<<< ori k_caches[0].shape=torch.Size([1, 128, 4, 128])
[2025-02-20 18:13:27,996] [92233] [281471198513440] [llm] [INFO] [flash_causal_qwen2.py-394] : <<<<<<<after transdata k_caches[0].shape=torch.Size([1, 128, 4, 128])
[2025-02-20 18:13:27,997] [92233] [281471198513440] [llm] [INFO] [logging.py-331] : >>>>>>id of kcache is 281470735710256 id of vcache is 281470735710832
[2025-02-20 18:13:28.392003] [error] [92233] [reshape_and_cache_operation.cpp:288] ReshapeAndCacheOperation_3_0_3 The blockSize of keyCache and valueCache should be 16-aligned
[2025-02-20 18:13:28.392055] [error] [92233] [operation_base.cpp:267] ReshapeAndCacheOperation_3_0_3 infer shape check fail, error code: 8
[2025-02-20 18:13:28.392063] [error] [92233] [graph_runner.cpp:662] AttentionRunner_3_0:0 node[3] infer shape fail, error code: 8
[2025-02-20 18:13:28.392076] [error] [92233] [graph_runner.cpp:263] AttentionRunner_3_0:0 node[3] setup fail, PreparseNodeVariantPack fail, error code:8
[2025-02-20 18:13:28.392084] [error] [92233] [graph_runner.cpp:762] Prefill_layerRunner_3:0 node[0] setup fail, error code:8
[2025-02-20 18:13:28.392090] [error] [92233] [graph_runner.cpp:270] Prefill_layerRunner_3:0 node[0] SetupNodeRunners fail, error code:8
[2025-02-20 18:13:28.392096] [error] [92233] [operation_base.cpp:462] Prefill_layer_3 runner setup fail
[2025-02-20 18:13:28.392102] [error] [92233] [operation_base.cpp:518] Prefill_layer_3 setup fail, error code: 8
Traceback (most recent call last):
File "/data/et_llm/vllm_npu/examples/test_offline.py", line 13, in <module>
llm = LLM(model=model_path,
^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.11/site-packages/vllm-0.3.3-py3.11.egg/vllm/entrypoints/llm.py", line 109, in __init__
self.llm_engine = LLMEngine.from_engine_args(engine_args)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.11/site-packages/vllm-0.3.3-py3.11.egg/vllm/engine/llm_engine.py", line 391, in from_engine_args
engine = cls(*engine_configs,
^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.11/site-packages/vllm-0.3.3-py3.11.egg/vllm/engine/llm_engine.py", line 131, in __init__
self._init_cache()
File "/usr/local/lib/python3.11/site-packages/vllm-0.3.3-py3.11.egg/vllm/engine/llm_engine.py", line 343, in _init_cache
num_blocks = self._run_workers(
^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.11/site-packages/vllm-0.3.3-py3.11.egg/vllm/engine/llm_engine.py", line 1041, in _run_workers
driver_worker_output = getattr(self.driver_worker,
^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib64/python3.11/site-packages/torch/utils/_contextlib.py", line 115, in decorate_context
return func(*args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.11/site-packages/vllm_npu-0.3.3-py3.11.egg/vllm_npu/worker/ascend_worker.py", line 104, in profile_num_available_blocks
File "/usr/local/lib64/python3.11/site-packages/torch/utils/_contextlib.py", line 115, in decorate_context
return func(*args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.11/site-packages/vllm_npu-0.3.3-py3.11.egg/vllm_npu/worker/model_runner.py", line 267, in profile_run
File "/usr/local/lib64/python3.11/site-packages/torch/utils/_contextlib.py", line 115, in decorate_context
return func(*args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.11/site-packages/vllm_npu-0.3.3-py3.11.egg/vllm_npu/worker/model_runner.py", line 232, in execute_model
File "/usr/local/lib64/python3.11/site-packages/torch/nn/modules/module.py", line 1518, in _wrapped_call_impl
return self._call_impl(*args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib64/python3.11/site-packages/torch/nn/modules/module.py", line 1527, in _call_impl
return forward_call(*args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.11/site-packages/vllm_npu-0.3.3-py3.11.egg/vllm_npu/model_executor/models/ascend/mindie_llm_wrapper.py", line 46, in forward
File "/usr/local/lib/python3.11/site-packages/mindie_llm/text_generator/adapter/generator_torch.py", line 127, in forward_tensor
logits = self.model_wrapper.forward_tensor(
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.11/site-packages/mindie_llm/modeling/model_wrapper/atb/atb_model_wrapper.py", line 116, in forward_tensor
logits = self.model_runner.forward(
^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/Ascend/atb-models/atb_llm/runner/model_runner.py", line 193, in forward
return self.model.forward(**kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/Ascend/atb-models/atb_llm/models/base/flash_causal_lm.py", line 458, in forward
logits = self.execute_ascend_operator(acl_inputs, acl_param, is_prefill)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/Ascend/atb-models/atb_llm/models/qwen2/flash_causal_qwen2.py", line 374, in execute_ascend_operator
acl_model_out = model_operation.execute(acl_inputs, acl_param)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
RuntimeError: Setup fail, enable log: export ASDOPS_LOG_LEVEL=ERROR, export ASDOPS_LOG_TO_STDOUT=1 to find the first error. For more details, see the MindIE official document
------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
python相关库版本如下：