使用的 Atlas 300I Pro 卡，驱动固件正常安装，npu-smi info 也可以读到信息
但是安装 torch 和 torch_npu 后调用就会报如下错误
>>> import torch
>>> import torch_npu
>>> a = torch.rand(3,4).npu()
[W compiler_depend.ts:623] Warning: expandable_segments currently defaults to false. You can enable this feature by `export PYTORCH_NPU_ALLOC_CONF = expandable_segments:True`. (function operator())
>>> b = a+a
>>> print(b)
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "/usr/local/lib/python3.8/dist-packages/torch/_tensor.py", line 431, in __repr__
return torch._tensor_str._str(self, tensor_contents=tensor_contents)
File "/usr/local/lib/python3.8/dist-packages/torch/_tensor_str.py", line 664, in _str
return _str_intern(self, tensor_contents=tensor_contents)
File "/usr/local/lib/python3.8/dist-packages/torch/_tensor_str.py", line 595, in _str_intern
tensor_str = _tensor_str(self, indent)
File "/usr/local/lib/python3.8/dist-packages/torch/_tensor_str.py", line 347, in _tensor_str
formatter = _Formatter(get_summarized_data(self) if summarize else self)
File "/usr/local/lib/python3.8/dist-packages/torch/_tensor_str.py", line 137, in __init__
nonzero_finite_vals = torch.masked_select(
RuntimeError: InnerRun:build/CMakeFiles/torch_npu.dir/compiler_depend.ts:210 OPS function error: MaskedSelect, error code is 500002
[ERROR] 2025-02-28-11:17:21 (PID:8840, Device:0, RankID:-1) ERR01100 OPS call acl api failed
[Error]: A GE error occurs in the system.
Rectify the fault based on the error information in the ascend log.
EZ9999: Inner Error!
EZ9999: 2025-02-28-11:17:21.777.612 The error from device(0), serial number is 1, there is an aicore error, core id is 0, error code = 0x800000, dump info: pc start: 0x800124000000000, current: 0x124000000118, vec error info: 0x1d5e3abe, mte error info: 0x300002b, ifu error info: 0x37bf9fc7e7780, ccu error info: 0xb6fc78cc006cffed, cube error info: 0x9d, biu error info: 0, aic error mask: 0x65000200d00028c, para base: 0x12c040261000, errorStr: The DDR address of the MTE instruction is out of range.[FUNC:PrintCoreErrorInfo][FILE:device_error_proc.cc][LINE:532]
TraceBack (most recent call last):
The extend info from device(0), serial number is 1, there is aicore error, core id is 0, aicore int: 0x10, aicore error2: 0, axi clamp ctrl: 0, axi clamp state: 0x1717, biu status0: 0x101d14000000000, biu status1: 0x80000201020000, clk gate mask: 0, dbg addr: 0, ecc en: 0, mte ccu ecc 1bit error: 0x2780000000000000, vector cube ecc 1bit error: 0, run stall: 0x1, dbg data0: 0, dbg data1: 0, dbg data2: 0, dbg data3: 0, dfx data: 0x6d[FUNC:PrintCoreErrorInfo][FILE:device_error_proc.cc][LINE:563]
The dha(mata) info from device(0), dha id is 0, dha status 1 info:0x23[FUNC:ProcessCoreErrorInfo][FILE:device_error_proc.cc][LINE:653]
The dha(mata) info from device(0), dha id is 1, dha status 1 info:0x83[FUNC:ProcessCoreErrorInfo][FILE:device_error_proc.cc][LINE:653]
The device(0), core list[0-0], error code is:[FUNC:PrintCoreInfoErrMsg][FILE:device_error_proc.cc][LINE:586]
coreId( 0): 0x800000 [FUNC:PrintCoreInfoErrMsg][FILE:device_error_proc.cc][LINE:600]
Kernel task happen error, retCode=0x26, [aicore exception].[FUNC:PreCheckTaskErr][FILE:davinic_kernel_task.cc][LINE:1220]
Op execute failed. origin_op_name [Add1], op_name [Add1], error_info: task_id 2, stream_id 3, tid 8840, device_id 0, retcode 0x7bc87[FUNC:ErrorTrackingCallback][FILE:error_tracking.cc][LINE:100]
Aicore kernel execute failed, device_id=0, stream_id=3, report_stream_id=3, task_id=2, flip_num=0, fault kernel_name=0/1_-1_0_Add1_tvmbin, fault kernel info ext=te_add_0c615903a3d4a33f38b4141663f091d05a24440264c8c03ce1dd587ae7e4db1a_1__kernel0, program id=0, hash=15795894736451304642.[FUNC:GetError][FILE:stream.cc][LINE:1082]
[AIC_INFO] after execute:args print end[FUNC:GetError][FILE:stream.cc][LINE:1082]
rtStreamSynchronizeWithTimeout execute failed, reason=[aicore exception][FUNC:FuncErrorReason][FILE:error_message_manage.cc][LINE:53]
Assert ((rt_ret) == 0) failed[FUNC:DoRtStreamSyncWithTimeout][FILE:utils.cc][LINE:45]
[Exec][Op]Execute op failed. op type = MaskedSelect, ge result = 1343225857[FUNC:ReportCallError][FILE:log_inner.cpp][LINE:161]
DEVICE[0] PID[8840]:
EXCEPTION STREAM:
Exception info:TGID=8840, model id=65535, stream id=3, stream phase=SCHEDULE
Message info[0]:RTS_HWTS: Aicore exception, slot_id=16, stream_id=3
Other info[0]:time=2025-02-28-11:17:11.476.803, function=process_hwts_error_exception, line=646, error code=0x26
主机系统为银河麒麟，替换为 openeuler22.03 后安装同样的环境就可以正常使用，请问是什么原因
点赞
收藏
回复
分享
举报
已解决
使用的 Atlas 300I Pro 卡，驱动固件正常安装，npu-smi info 也可以读到信息
但是安装 torch 和 torch_npu 后调用就会报如下错误
>>> import torch
>>> import torch_npu
>>> a = torch.rand(3,4).npu()
[W compiler_depend.ts:623] Warning: expandable_segments currently defaults to false. You can enable this feature by `export PYTORCH_NPU_ALLOC_CONF = expandable_segments:True`. (function operator())
>>> b = a+a
>>> print(b)
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "/usr/local/lib/python3.8/dist-packages/torch/_tensor.py", line 431, in __repr__
return torch._tensor_str._str(self, tensor_contents=tensor_contents)
File "/usr/local/lib/python3.8/dist-packages/torch/_tensor_str.py", line 664, in _str
return _str_intern(self, tensor_contents=tensor_contents)
File "/usr/local/lib/python3.8/dist-packages/torch/_tensor_str.py", line 595, in _str_intern
tensor_str = _tensor_str(self, indent)
File "/usr/local/lib/python3.8/dist-packages/torch/_tensor_str.py", line 347, in _tensor_str
formatter = _Formatter(get_summarized_data(self) if summarize else self)
File "/usr/local/lib/python3.8/dist-packages/torch/_tensor_str.py", line 137, in __init__
nonzero_finite_vals = torch.masked_select(
RuntimeError: InnerRun:build/CMakeFiles/torch_npu.dir/compiler_depend.ts:210 OPS function error: MaskedSelect, error code is 500002
[ERROR] 2025-02-28-11:17:21 (PID:8840, Device:0, RankID:-1) ERR01100 OPS call acl api failed
[Error]: A GE error occurs in the system.
Rectify the fault based on the error information in the ascend log.
EZ9999: Inner Error!
EZ9999: 2025-02-28-11:17:21.777.612 The error from device(0), serial number is 1, there is an aicore error, core id is 0, error code = 0x800000, dump info: pc start: 0x800124000000000, current: 0x124000000118, vec error info: 0x1d5e3abe, mte error info: 0x300002b, ifu error info: 0x37bf9fc7e7780, ccu error info: 0xb6fc78cc006cffed, cube error info: 0x9d, biu error info: 0, aic error mask: 0x65000200d00028c, para base: 0x12c040261000, errorStr: The DDR address of the MTE instruction is out of range.[FUNC:PrintCoreErrorInfo][FILE:device_error_proc.cc][LINE:532]
TraceBack (most recent call last):
The extend info from device(0), serial number is 1, there is aicore error, core id is 0, aicore int: 0x10, aicore error2: 0, axi clamp ctrl: 0, axi clamp state: 0x1717, biu status0: 0x101d14000000000, biu status1: 0x80000201020000, clk gate mask: 0, dbg addr: 0, ecc en: 0, mte ccu ecc 1bit error: 0x2780000000000000, vector cube ecc 1bit error: 0, run stall: 0x1, dbg data0: 0, dbg data1: 0, dbg data2: 0, dbg data3: 0, dfx data: 0x6d[FUNC:PrintCoreErrorInfo][FILE:device_error_proc.cc][LINE:563]
The dha(mata) info from device(0), dha id is 0, dha status 1 info:0x23[FUNC:ProcessCoreErrorInfo][FILE:device_error_proc.cc][LINE:653]
The dha(mata) info from device(0), dha id is 1, dha status 1 info:0x83[FUNC:ProcessCoreErrorInfo][FILE:device_error_proc.cc][LINE:653]
The device(0), core list[0-0], error code is:[FUNC:PrintCoreInfoErrMsg][FILE:device_error_proc.cc][LINE:586]
coreId( 0): 0x800000 [FUNC:PrintCoreInfoErrMsg][FILE:device_error_proc.cc][LINE:600]
Kernel task happen error, retCode=0x26, [aicore exception].[FUNC:PreCheckTaskErr][FILE:davinic_kernel_task.cc][LINE:1220]
Op execute failed. origin_op_name [Add1], op_name [Add1], error_info: task_id 2, stream_id 3, tid 8840, device_id 0, retcode 0x7bc87[FUNC:ErrorTrackingCallback][FILE:error_tracking.cc][LINE:100]
Aicore kernel execute failed, device_id=0, stream_id=3, report_stream_id=3, task_id=2, flip_num=0, fault kernel_name=0/1_-1_0_Add1_tvmbin, fault kernel info ext=te_add_0c615903a3d4a33f38b4141663f091d05a24440264c8c03ce1dd587ae7e4db1a_1__kernel0, program id=0, hash=15795894736451304642.[FUNC:GetError][FILE:stream.cc][LINE:1082]
[AIC_INFO] after execute:args print end[FUNC:GetError][FILE:stream.cc][LINE:1082]
rtStreamSynchronizeWithTimeout execute failed, reason=[aicore exception][FUNC:FuncErrorReason][FILE:error_message_manage.cc][LINE:53]
Assert ((rt_ret) == 0) failed[FUNC:DoRtStreamSyncWithTimeout][FILE:utils.cc][LINE:45]
[Exec][Op]Execute op failed. op type = MaskedSelect, ge result = 1343225857[FUNC:ReportCallError][FILE:log_inner.cpp][LINE:161]
DEVICE[0] PID[8840]:
EXCEPTION STREAM:
Exception info:TGID=8840, model id=65535, stream id=3, stream phase=SCHEDULE
Message info[0]:RTS_HWTS: Aicore exception, slot_id=16, stream_id=3
Other info[0]:time=2025-02-28-11:17:11.476.803, function=process_hwts_error_exception, line=646, error code=0x26
主机系统为银河麒麟，替换为 openeuler22.03 后安装同样的环境就可以正常使用，请问是什么原因