NonZero
Traceback (most recent call last):
File "grounded_sam_demo.py", line 190, in <module>
model, image, text_prompt, box_threshold, text_threshold, device=device
File "grounded_sam_demo.py", line 63, in get_grounding_output
outputs = model(image[None], captions=[caption])
File "/home/ma-user/anaconda3/envs/python-3.7.10/lib/python3.7/site-packages/torch/nn/modules/module.py", line 1110, in _call_impl
return forward_call(*input, **kwargs)
File "/home/ma-user/work/Grounded-Segment-Anything-main/GroundingDINO/groundingdino/models/GroundingDINO/groundingdino.py", line 242, in forward
tokenized, self.specical_tokens, self.tokenizer
File "/home/ma-user/work/Grounded-Segment-Anything-main/GroundingDINO/groundingdino/models/GroundingDINO/bertwarper.py", line 240, in generate_masks_with_special_tokens_and_transfer_map
idxs = torch.nonzero(special_tokens_mask)
RuntimeError: Run:/usr1/workspace/FPTA_Daily_open_pytorchv1.11.0-5.0.rc1/CODE/torch_npu/csrc/framework/OpParamMaker.cpp:138 NPU error,NPU error code is:507018
E39999: Inner Error!
E39999 An exception occurred during AICPU execution, stream_id:2, task_id:18, errcode:21008, msg:inner error[FUNC:ProcessAicpuErrorInfo][FILE:device_error_proc.cc][LINE:667]
Aicpu kernel execute failed, device_id=0, stream_id=2, task_id=18.[FUNC:PrintAicpuErrorInfo][FILE:task.cc][LINE:747]
Aicpu kernel execute failed, device_id=0, stream_id=2, task_id=18, fault op_name=NonZero[FUNC:GetError][FILE:stream.cc][LINE:846]
rtStreamSynchronize execute failed, reason=[aicpu exception][FUNC:FuncErrorReason][FILE:error_message_manage.cc][LINE:45]
Call rtStreamSynchronize(stream) fail, ret: 0x7BC8A[FUNC:LaunchKernel][FILE:op_task.cc][LINE:1403]
[Exec][Op]Execute op failed, ge result = 507018[FUNC:ReportCallError][FILE:log_inner.cpp][LINE:162]
An exception occurred during AICPU execution, stream_id:2, task_id:21, errcode:21008, msg:inner error[FUNC:ProcessAicpuErrorInfo][FILE:device_error_proc.cc][LINE:667]
Aicpu kernel execute failed, device_id=0, stream_id=2, task_id=21.[FUNC:PrintAicpuErrorInfo][FILE:task.cc][LINE:747]
Aicpu kernel execute failed, device_id=0, stream_id=2, task_id=21, fault op_name=NonZero[FUNC:GetError][FILE:stream.cc][LINE:846]
An exception occurred during AICPU execution, stream_id:2, task_id:24, errcode:21008, msg:inner error[FUNC:ProcessAicpuErrorInfo][FILE:device_error_proc.cc][LINE:667]
Aicpu kernel execute failed, device_id=0, stream_id=2, task_id=24.[FUNC:PrintAicpuErrorInfo][FILE:task.cc][LINE:747]
Aicpu kernel execute failed, device_id=0, stream_id=2, task_id=24, fault op_name=NonZero[FUNC:GetError][FILE:stream.cc][LINE:846]
DEVICE[0] PID[37626]:
EXCEPTION TASK:
Exception info:TGID=109318, model id=65535, stream id=2, stream phase=3, task id=18, task type=aicpu kernel, recently received task id=19, recently send task id=17, task phase=RUN
Message info[0]:aicpu=0,slot_id=0,report_mailbox_flag=0x5a5a5a5a,state=0x5210
Other info[0]:time=2023-08-29-04:40:17.785.861, function=proc_aicpu_task_done, line=972, error code=0x2a
EXCEPTION TASK:
Exception info:TGID=109318, model id=65535, stream id=2, stream phase=3, task id=21, task type=aicpu kernel, recently received task id=22, recently send task id=20, task phase=RUN
Message info[0]:aicpu=0,slot_id=0,report_mailbox_flag=0x5a5a5a5a,state=0x5210
Other info[0]:time=2023-08-29-04:40:17.790.300, function=proc_aicpu_task_done, line=972, error code=0x2a
EXCEPTION TASK:
Exception info:TGID=109318, model id=65535, stream id=2, stream phase=3, task id=24, task type=aicpu kernel, recently received task id=25, recently send task id=23, task phase=RUN
Message info[0]:aicpu=0,slot_id=0,report_mailbox_flag=0x5a5a5a5a,state=0x5210
Other info[0]:time=2023-08-29-04:40:17.793.767, function=proc_aicpu_task_done, line=972, error code=0x2a
THPModule_npu_shutdown success.
点赞
收藏
回复
分享
举报
已解决
问题描述：我是使用学校提供的华为账号，使用的是昇腾的服务器Ascend 910 CPU24核 内存96GiB。通过modelarts启动jupyter notebook来运行代码，但是在运行的过程中，总是有以下错误，求大佬帮忙解决一下，我是深度学习小白，而且又刚接触昇腾平台，球球了
这是报错信息
NonZero Traceback (most recent call last): File "grounded_sam_demo.py", line 190, in <module> model, image, text_prompt, box_threshold, text_threshold, device=device File "grounded_sam_demo.py", line 63, in get_grounding_output outputs = model(image[None], captions=[caption]) File "/home/ma-user/anaconda3/envs/python-3.7.10/lib/python3.7/site-packages/torch/nn/modules/module.py", line 1110, in _call_impl return forward_call(*input, **kwargs) File "/home/ma-user/work/Grounded-Segment-Anything-main/GroundingDINO/groundingdino/models/GroundingDINO/groundingdino.py", line 242, in forward tokenized, self.specical_tokens, self.tokenizer File "/home/ma-user/work/Grounded-Segment-Anything-main/GroundingDINO/groundingdino/models/GroundingDINO/bertwarper.py", line 240, in generate_masks_with_special_tokens_and_transfer_map idxs = torch.nonzero(special_tokens_mask) RuntimeError: Run:/usr1/workspace/FPTA_Daily_open_pytorchv1.11.0-5.0.rc1/CODE/torch_npu/csrc/framework/OpParamMaker.cpp:138 NPU error,NPU error code is:507018 E39999: Inner Error! E39999 An exception occurred during AICPU execution, stream_id:2, task_id:18, errcode:21008, msg:inner error[FUNC:ProcessAicpuErrorInfo][FILE:device_error_proc.cc][LINE:667] Aicpu kernel execute failed, device_id=0, stream_id=2, task_id=18.[FUNC:PrintAicpuErrorInfo][FILE:task.cc][LINE:747] Aicpu kernel execute failed, device_id=0, stream_id=2, task_id=18, fault op_name=NonZero[FUNC:GetError][FILE:stream.cc][LINE:846] rtStreamSynchronize execute failed, reason=[aicpu exception][FUNC:FuncErrorReason][FILE:error_message_manage.cc][LINE:45] Call rtStreamSynchronize(stream) fail, ret: 0x7BC8A[FUNC:LaunchKernel][FILE:op_task.cc][LINE:1403] [Exec][Op]Execute op failed, ge result = 507018[FUNC:ReportCallError][FILE:log_inner.cpp][LINE:162] An exception occurred during AICPU execution, stream_id:2, task_id:21, errcode:21008, msg:inner error[FUNC:ProcessAicpuErrorInfo][FILE:device_error_proc.cc][LINE:667] Aicpu kernel execute failed, device_id=0, stream_id=2, task_id=21.[FUNC:PrintAicpuErrorInfo][FILE:task.cc][LINE:747] Aicpu kernel execute failed, device_id=0, stream_id=2, task_id=21, fault op_name=NonZero[FUNC:GetError][FILE:stream.cc][LINE:846] An exception occurred during AICPU execution, stream_id:2, task_id:24, errcode:21008, msg:inner error[FUNC:ProcessAicpuErrorInfo][FILE:device_error_proc.cc][LINE:667] Aicpu kernel execute failed, device_id=0, stream_id=2, task_id=24.[FUNC:PrintAicpuErrorInfo][FILE:task.cc][LINE:747] Aicpu kernel execute failed, device_id=0, stream_id=2, task_id=24, fault op_name=NonZero[FUNC:GetError][FILE:stream.cc][LINE:846] DEVICE[0] PID[37626]: EXCEPTION TASK: Exception info:TGID=109318, model id=65535, stream id=2, stream phase=3, task id=18, task type=aicpu kernel, recently received task id=19, recently send task id=17, task phase=RUN Message info[0]:aicpu=0,slot_id=0,report_mailbox_flag=0x5a5a5a5a,state=0x5210 Other info[0]:time=2023-08-29-04:40:17.785.861, function=proc_aicpu_task_done, line=972, error code=0x2a EXCEPTION TASK: Exception info:TGID=109318, model id=65535, stream id=2, stream phase=3, task id=21, task type=aicpu kernel, recently received task id=22, recently send task id=20, task phase=RUN Message info[0]:aicpu=0,slot_id=0,report_mailbox_flag=0x5a5a5a5a,state=0x5210 Other info[0]:time=2023-08-29-04:40:17.790.300, function=proc_aicpu_task_done, line=972, error code=0x2a EXCEPTION TASK: Exception info:TGID=109318, model id=65535, stream id=2, stream phase=3, task id=24, task type=aicpu kernel, recently received task id=25, recently send task id=23, task phase=RUN Message info[0]:aicpu=0,slot_id=0,report_mailbox_flag=0x5a5a5a5a,state=0x5210 Other info[0]:time=2023-08-29-04:40:17.793.767, function=proc_aicpu_task_done, line=972, error code=0x2a THPModule_npu_shutdown success.