问题描述：在Atlas 300V设备上推理ModelZoo中git的Janus-pro中的Inference.py时出现一个RuntimeError: native_layer_norm:build/CMakeFiles/torch_npu.dir/compiler_depend.ts:92 NPU function error: call aclnnLayerNorm failed, error code is 561103的报错
硬件：Atlas 300V
CANN版本:8.2.RC1
MindIE版本：2.1.RC1
torch版本：2.1.0
torch_npu版本：2.1.0.post13
python版本：3.10.0
详细报错：
python inference.py --path /home/wyg/project/Janus-Pro-1B --device_id 0 --type fp16
/home/wyg/miniconda3/envs/py310/lib/python3.10/site-packages/torch_npu/contrib/transfer_to_npu.py:302: ImportWarning:
*************************************************************************************************************
The torch.Tensor.cuda and torch.nn.Module.cuda are replaced with torch.Tensor.npu and torch.nn.Module.npu now..
The torch.cuda.DoubleTensor is replaced with torch.npu.FloatTensor cause the double type is not supported now..
The backend in torch.distributed.init_process_group set to hccl now..
The torch.cuda.* and torch.cuda.amp.* are replaced with torch.npu.* and torch.npu.amp.* now..
The device parameters have been replaced with npu in the function below:
torch.logspace, torch.randint, torch.hann_window, torch.rand, torch.full_like, torch.ones_like, torch.rand_like, torch.randperm, torch.arange, torch.frombuffer, torch.normal, torch._empty_per_channel_affine_quantized, torch.empty_strided, torch.empty_like, torch.scalar_tensor, torch.tril_indices, torch.bartlett_window, torch.ones, torch.sparse_coo_tensor, torch.randn, torch.kaiser_window, torch.tensor, torch.triu_indices, torch.as_tensor, torch.zeros, torch.randint_like, torch.full, torch.eye, torch._sparse_csr_tensor_unsafe, torch.empty, torch._sparse_coo_tensor_unsafe, torch.blackman_window, torch.zeros_like, torch.range, torch.sparse_csr_tensor, torch.randn_like, torch.from_file, torch._cudnn_init_dropout_state, torch._empty_affine_quantized, torch.linspace, torch.hamming_window, torch.empty_quantized, torch._pin_memory, torch.autocast, torch.load, torch.set_default_device, torch.Tensor.new_empty, torch.Tensor.new_empty_strided, torch.Tensor.new_full, torch.Tensor.new_ones, torch.Tensor.new_tensor, torch.Tensor.new_zeros, torch.Tensor.to, torch.Tensor.pin_memory, torch.nn.Module.to, torch.nn.Module.to_empty
*************************************************************************************************************
warnings.warn(msg, ImportWarning)
/home/wyg/miniconda3/envs/py310/lib/python3.10/site-packages/torch_npu/contrib/transfer_to_npu.py:257: RuntimeWarning: torch.jit.script and torch.jit.script_method will be disabled by transfer_to_npu, which currently does not support them, if you need to enable them, please do not use transfer_to_npu.
warnings.warn(msg, RuntimeWarning)
Python version is above 3.10, patching the collections module.
Special tokens have been added in the vocabulary, make sure the associated word embeddings are fine-tuned or trained.
/home/wyg/miniconda3/envs/py310/lib/python3.10/site-packages/torch/_utils.py:831: UserWarning: TypedStorage is deprecated. It will be removed in the future and UntypedStorage will be the only storage class. This should only matter to you if you are using storages directly. To access UntypedStorage directly, use tensor.untyped_storage() instead of tensor.storage()
return self.fget.__get__(instance, owner)()
Traceback (most recent call last):
File "/home/wyg/project/ModelZoo-PyTorch/MindIE/MultiModal/Janus-Pro/inference.py", line 86, in <module>
inputs_embeds = vl_gpt.prepare_inputs_embeds(**prepare_inputs)
File "/home/wyg/project/ModelZoo-PyTorch/MindIE/MultiModal/Janus-Pro/janus/models/modeling_vlm.py", line 246, in prepare_inputs_embeds
images_embeds = self.aligner(self.vision_model(images))
File "/home/wyg/miniconda3/envs/py310/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1518, in _wrapped_call_impl
return self._call_impl(*args, **kwargs)
File "/home/wyg/miniconda3/envs/py310/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1527, in _call_impl
return forward_call(*args, **kwargs)
File "/home/wyg/project/ModelZoo-PyTorch/MindIE/MultiModal/Janus-Pro/janus/models/clip_encoder.py", line 120, in forward
image_forward_outs = self.vision_tower(images, **self.forward_kwargs)
File "/home/wyg/miniconda3/envs/py310/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1518, in _wrapped_call_impl
return self._call_impl(*args, **kwargs)
File "/home/wyg/miniconda3/envs/py310/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1527, in _call_impl
return forward_call(*args, **kwargs)
File "/home/wyg/project/ModelZoo-PyTorch/MindIE/MultiModal/Janus-Pro/janus/models/siglip_vit.py", line 589, in forward
x = self.forward_features(x)
File "/home/wyg/project/ModelZoo-PyTorch/MindIE/MultiModal/Janus-Pro/janus/models/siglip_vit.py", line 573, in forward_features
x = self.blocks(x)
File "/home/wyg/miniconda3/envs/py310/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1518, in _wrapped_call_impl
return self._call_impl(*args, **kwargs)
File "/home/wyg/miniconda3/envs/py310/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1527, in _call_impl
return forward_call(*args, **kwargs)
File "/home/wyg/miniconda3/envs/py310/lib/python3.10/site-packages/torch/nn/modules/container.py", line 215, in forward
input = module(input)
File "/home/wyg/miniconda3/envs/py310/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1518, in _wrapped_call_impl
return self._call_impl(*args, **kwargs)
File "/home/wyg/miniconda3/envs/py310/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1527, in _call_impl
return forward_call(*args, **kwargs)
File "/home/wyg/project/ModelZoo-PyTorch/MindIE/MultiModal/Janus-Pro/janus/models/siglip_vit.py", line 254, in forward
x = x + self.drop_path1(self.ls1(self.attn(self.norm1(x))))
File "/home/wyg/miniconda3/envs/py310/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1518, in _wrapped_call_impl
return self._call_impl(*args, **kwargs)
File "/home/wyg/miniconda3/envs/py310/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1527, in _call_impl
return forward_call(*args, **kwargs)
File "/home/wyg/miniconda3/envs/py310/lib/python3.10/site-packages/torch/nn/modules/normalization.py", line 196, in forward
return F.layer_norm(
File "/home/wyg/miniconda3/envs/py310/lib/python3.10/site-packages/torch/nn/functional.py", line 2543, in layer_norm
return torch.layer_norm(input, normalized_shape, weight, bias, eps, torch.backends.cudnn.enabled)
RuntimeError: native_layer_norm:build/CMakeFiles/torch_npu.dir/compiler_depend.ts:92 NPU function error: call aclnnLayerNorm failed, error code is 561103
[ERROR] 2025-09-12-09:26:00 (PID:3257960, Device:0, RankID:-1) ERR00100 PTA call acl api failed.
EZ9999: Inner Error!
EZ9999: [PID: 3257960] 2025-09-12-09:26:00.257.593 Parse dynamic kernel config fail.
TraceBack (most recent call last):
AclOpKernelInit failed opType
Cast ADD_TO_LAUNCHER_LIST_AICORE failed.
请大大们帮忙看看，万分感谢~
点赞
收藏
回复
分享
举报
已解决
问题描述：在Atlas 300V设备上推理ModelZoo中git的Janus-pro中的Inference.py时出现一个RuntimeError: native_layer_norm:build/CMakeFiles/torch_npu.dir/compiler_depend.ts:92 NPU function error: call aclnnLayerNorm failed, error code is 561103的报错
硬件：Atlas 300V
CANN版本:8.2.RC1
MindIE版本：2.1.RC1
torch版本：2.1.0
torch_npu版本：2.1.0.post13
python版本：3.10.0
详细报错：
python inference.py --path /home/wyg/project/Janus-Pro-1B --device_id 0 --type fp16
/home/wyg/miniconda3/envs/py310/lib/python3.10/site-packages/torch_npu/contrib/transfer_to_npu.py:302: ImportWarning:
*************************************************************************************************************
The torch.Tensor.cuda and torch.nn.Module.cuda are replaced with torch.Tensor.npu and torch.nn.Module.npu now..
The torch.cuda.DoubleTensor is replaced with torch.npu.FloatTensor cause the double type is not supported now..
The backend in torch.distributed.init_process_group set to hccl now..
The torch.cuda.* and torch.cuda.amp.* are replaced with torch.npu.* and torch.npu.amp.* now..
The device parameters have been replaced with npu in the function below:
torch.logspace, torch.randint, torch.hann_window, torch.rand, torch.full_like, torch.ones_like, torch.rand_like, torch.randperm, torch.arange, torch.frombuffer, torch.normal, torch._empty_per_channel_affine_quantized, torch.empty_strided, torch.empty_like, torch.scalar_tensor, torch.tril_indices, torch.bartlett_window, torch.ones, torch.sparse_coo_tensor, torch.randn, torch.kaiser_window, torch.tensor, torch.triu_indices, torch.as_tensor, torch.zeros, torch.randint_like, torch.full, torch.eye, torch._sparse_csr_tensor_unsafe, torch.empty, torch._sparse_coo_tensor_unsafe, torch.blackman_window, torch.zeros_like, torch.range, torch.sparse_csr_tensor, torch.randn_like, torch.from_file, torch._cudnn_init_dropout_state, torch._empty_affine_quantized, torch.linspace, torch.hamming_window, torch.empty_quantized, torch._pin_memory, torch.autocast, torch.load, torch.set_default_device, torch.Tensor.new_empty, torch.Tensor.new_empty_strided, torch.Tensor.new_full, torch.Tensor.new_ones, torch.Tensor.new_tensor, torch.Tensor.new_zeros, torch.Tensor.to, torch.Tensor.pin_memory, torch.nn.Module.to, torch.nn.Module.to_empty
*************************************************************************************************************
warnings.warn(msg, ImportWarning)
/home/wyg/miniconda3/envs/py310/lib/python3.10/site-packages/torch_npu/contrib/transfer_to_npu.py:257: RuntimeWarning: torch.jit.script and torch.jit.script_method will be disabled by transfer_to_npu, which currently does not support them, if you need to enable them, please do not use transfer_to_npu.
warnings.warn(msg, RuntimeWarning)
Python version is above 3.10, patching the collections module.
Special tokens have been added in the vocabulary, make sure the associated word embeddings are fine-tuned or trained.
/home/wyg/miniconda3/envs/py310/lib/python3.10/site-packages/torch/_utils.py:831: UserWarning: TypedStorage is deprecated. It will be removed in the future and UntypedStorage will be the only storage class. This should only matter to you if you are using storages directly. To access UntypedStorage directly, use tensor.untyped_storage() instead of tensor.storage()
return self.fget.__get__(instance, owner)()
Traceback (most recent call last):
File "/home/wyg/project/ModelZoo-PyTorch/MindIE/MultiModal/Janus-Pro/inference.py", line 86, in <module>
inputs_embeds = vl_gpt.prepare_inputs_embeds(**prepare_inputs)
File "/home/wyg/project/ModelZoo-PyTorch/MindIE/MultiModal/Janus-Pro/janus/models/modeling_vlm.py", line 246, in prepare_inputs_embeds
images_embeds = self.aligner(self.vision_model(images))
File "/home/wyg/miniconda3/envs/py310/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1518, in _wrapped_call_impl
return self._call_impl(*args, **kwargs)
File "/home/wyg/miniconda3/envs/py310/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1527, in _call_impl
return forward_call(*args, **kwargs)
File "/home/wyg/project/ModelZoo-PyTorch/MindIE/MultiModal/Janus-Pro/janus/models/clip_encoder.py", line 120, in forward
image_forward_outs = self.vision_tower(images, **self.forward_kwargs)
File "/home/wyg/miniconda3/envs/py310/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1518, in _wrapped_call_impl
return self._call_impl(*args, **kwargs)
File "/home/wyg/miniconda3/envs/py310/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1527, in _call_impl
return forward_call(*args, **kwargs)
File "/home/wyg/project/ModelZoo-PyTorch/MindIE/MultiModal/Janus-Pro/janus/models/siglip_vit.py", line 589, in forward
x = self.forward_features(x)
File "/home/wyg/project/ModelZoo-PyTorch/MindIE/MultiModal/Janus-Pro/janus/models/siglip_vit.py", line 573, in forward_features
x = self.blocks(x)
File "/home/wyg/miniconda3/envs/py310/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1518, in _wrapped_call_impl
return self._call_impl(*args, **kwargs)
File "/home/wyg/miniconda3/envs/py310/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1527, in _call_impl
return forward_call(*args, **kwargs)
File "/home/wyg/miniconda3/envs/py310/lib/python3.10/site-packages/torch/nn/modules/container.py", line 215, in forward
input = module(input)
File "/home/wyg/miniconda3/envs/py310/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1518, in _wrapped_call_impl
return self._call_impl(*args, **kwargs)
File "/home/wyg/miniconda3/envs/py310/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1527, in _call_impl
return forward_call(*args, **kwargs)
File "/home/wyg/project/ModelZoo-PyTorch/MindIE/MultiModal/Janus-Pro/janus/models/siglip_vit.py", line 254, in forward
x = x + self.drop_path1(self.ls1(self.attn(self.norm1(x))))
File "/home/wyg/miniconda3/envs/py310/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1518, in _wrapped_call_impl
return self._call_impl(*args, **kwargs)
File "/home/wyg/miniconda3/envs/py310/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1527, in _call_impl
return forward_call(*args, **kwargs)
File "/home/wyg/miniconda3/envs/py310/lib/python3.10/site-packages/torch/nn/modules/normalization.py", line 196, in forward
return F.layer_norm(
File "/home/wyg/miniconda3/envs/py310/lib/python3.10/site-packages/torch/nn/functional.py", line 2543, in layer_norm
return torch.layer_norm(input, normalized_shape, weight, bias, eps, torch.backends.cudnn.enabled)
RuntimeError: native_layer_norm:build/CMakeFiles/torch_npu.dir/compiler_depend.ts:92 NPU function error: call aclnnLayerNorm failed, error code is 561103
[ERROR] 2025-09-12-09:26:00 (PID:3257960, Device:0, RankID:-1) ERR00100 PTA call acl api failed.
EZ9999: Inner Error!
EZ9999: [PID: 3257960] 2025-09-12-09:26:00.257.593 Parse dynamic kernel config fail.
TraceBack (most recent call last):
AclOpKernelInit failed opType
Cast ADD_TO_LAUNCHER_LIST_AICORE failed.
请大大们帮忙看看，万分感谢~