运行平台信息如下:
NAME="Kylin" VERSION="银河麒麟桌面操作系统V10 SP1" VERSION_US="Kylin Linux Desktop V10 SP1" ID=kylin ID_LIKE=debian PRETTY_NAME="Kylin V10 SP1" VERSION_ID="v10" HOME_URL="http://www.kylinos.cn/" SUPPORT_URL="http://www.kylinos.cn/support/technology.html" BUG_REPORT_URL="http://www.kylinos.cn/" PRIVACY_POLICY_URL="http://www.kylinos.cn" VERSION_CODENAME=kylin UBUNTU_CODENAME=kylin PROJECT_CODENAME=V10SP1 KYLIN_RELEASE_ID="2503"
推理卡:300iduo×1
驱动版本:Ascend-hdk-310p-npu-driver_25.2.0_linux-aarch64
固件版本:Ascend-hdk-310p-npu-firmware_7.7.0.6.236
mindie版本:2.1.RC1-300I-Duo-py311-openeuler24.03-lts
模型信息:DeepSeek-R1-Distill-Qwen-1.5B
问题描述:使用相同硬件平台在其他操作系统时可以正常推理模型,但是在”银河麒麟桌面操作系统V10 SP1“下运行报错如下:
[W compiler_depend.ts:57] Warning: EZ9999: Inner Error!
EZ9999: [PID: 141] 2025-08-13-13:41:58.752.109 The error from device(1), serial number is 6, there is an error of aicore, core id is 5, error code = 0x800000, dump info: pc start: 0x8001242000000f0, current: 0x1242000001b8, vec error info: 0x1fbefefd, mte error info: 0x300006b, ifu error info: 0x35e917b2f5b80, ccu error info: 0xbab19eed004f2047, cube error info: 0xbb, biu error info: 0, aic error mask: 0x65000200d00028c, para base: 0x12c000261000, errorStr: The DDR address of the MTE instruction is out of range.[FUNC:PrintCoreErrorInfo][FILE:device_error_core_proc.cc][LINE:46]
TraceBack (most recent call last):
The extend info from device(1), serial number is 6, there is aicore error, core id is 5, aicore int: 0x10, aicore error2: 0, axi clamp ctrl: 0, axi clamp state: 0x1717, biu status0: 0x101d14000000000, biu status1: 0x80000201020000, clk gate mask: 0, dbg addr: 0, ecc en: 0, mte ccu ecc 1bit error: 0x1700000000000000, vector cube ecc 1bit error: 0, run stall: 0x1, dbg data0: 0, dbg data1: 0, dbg data2: 0, dbg data3: 0, dfx data: 0xf6[FUNC:PrintCoreErrorInfo][FILE:device_error_core_proc.cc][LINE:77]
The dha(mata) info from device(1), dha id is 0, dha status 1 info:0x23[FUNC:ProcessCoreErrorInfo][FILE:device_error_core_proc.cc][LINE:167]
The dha(mata) info from device(1), dha id is 1, dha status 1 info:0x3[FUNC:ProcessCoreErrorInfo][FILE:device_error_core_proc.cc][LINE:167]
The device(1), core list[0-0], error code is:[FUNC:PrintCoreInfoErrMsg][FILE:device_error_core_proc.cc][LINE:100]
coreId( 0): 0x800000 [FUNC:PrintCoreInfoErrMsg][FILE:device_error_core_proc.cc][LINE:114]
Kernel task happen error, retCode=0x26, [aicore exception].[FUNC:PreCheckTaskErr][FILE:davinci_kernel_task.cc][LINE:1539]
[AIC_INFO] after execute:args print end[FUNC:GetError][FILE:stream.cc][LINE:1183]
Aicore kernel execute failed, device_id=1, stream_id=2, report_stream_id=2, task_id=2, flip_num=0, fault kernel_name=ZerosLike_d2db1a80c523e7e59a032c95969880af_high_performance_210000000, fault kernel info ext=none, program id=0, hash=3724941331859449102.[FUNC:GetError][FILE:stream.cc][LINE:1183]
rtStreamSynchronize execute failed, reason=[aicore exception][FUNC:FuncErrorReason][FILE:error_message_manage.cc][LINE:53]
synchronize stream failed, runtime result = 507015[FUNC:ReportCallError][FILE:log_inner.cpp][LINE:161]
DEVICE[1] PID[141]:
EXCEPTION STREAM:
Exception info:TGID=23427, model id=65535, stream id=2, stream phase=SCHEDULE
Message info[0]:RTS_HWTS: Aicore exception, slot_id=7, stream_id=2
Other info[0]:time=2025-08-13-13:41:58.064.062, function=process_hwts_error_exception, line=646, error code=0x26 (function copy_between_host_and_device_opapi)
[ERROR] TBE Subprocess[task_distribute] raise error[], main process disappeared!
Process ForkServerProcess-8:
Traceback (most recent call last):
File "/usr/lib64/python3.11/multiprocessing/managers.py", line 814, in _callmethod
conn = self._tls.connection
^^^^^^^^^^^^^^^^^^^^
AttributeError: 'ForkAwareLocal' object has no attribute 'connection'
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/usr/lib64/python3.11/multiprocessing/process.py", line 314, in _bootstrap
self.run()
File "/usr/lib64/python3.11/multiprocessing/process.py", line 108, in run
self._target(*self._args, **self._kwargs)
File "/usr/local/Ascend/ascend-toolkit/latest/python/site-packages/tbe/common/repository_manager/route.py", line 71, in wrapper
raise exp
File "/usr/local/Ascend/ascend-toolkit/latest/python/site-packages/tbe/common/repository_manager/route.py", line 63, in wrapper
func(*args, **kwargs)
File "/usr/local/Ascend/ascend-toolkit/latest/python/site-packages/tbe/common/repository_manager/route.py", line 264, in task_distribute
resource_proxy[SUB_PROCESS_STATE].append(True)
File "<string>", line 2, in append
File "/usr/lib64/python3.11/multiprocessing/managers.py", line 818, in _callmethod
self._connect()
File "/usr/lib64/python3.11/multiprocessing/managers.py", line 805, in _connect
conn = self._Client(self._token.address, authkey=self._authkey)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/lib64/python3.11/multiprocessing/connection.py", line 518, in Client
c = SocketClient(address)
^^^^^^^^^^^^^^^^^^^^^
File "/usr/lib64/python3.11/multiprocessing/connection.py", line 646, in SocketClient
s.connect(address)
ConnectionRefusedError: [Errno 111] Connection refused
Process ForkServerProcess-3:
Traceback (most recent call last):
File "/usr/lib64/python3.11/multiprocessing/managers.py", line 814, in _callmethod
conn = self._tls.connection
^^^^^^^^^^^^^^^^^^^^
AttributeError: 'ForkAwareLocal' object has no attribute 'connection'
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/usr/lib64/python3.11/multiprocessing/process.py", line 314, in _bootstrap
self.run()
File "/usr/lib64/python3.11/multiprocessing/process.py", line 108, in run
self._target(*self._args, **self._kwargs)
File "/usr/local/Ascend/ascend-toolkit/latest/python/site-packages/tbe/common/repository_manager/route.py", line 71, in wrapper
raise exp
File "/usr/local/Ascend/ascend-toolkit/latest/python/site-packages/tbe/common/repository_manager/route.py", line 63, in wrapper
func(*args, **kwargs)
File "/usr/local/Ascend/ascend-toolkit/latest/python/site-packages/tbe/common/repository_manager/route.py", line 264, in task_distribute
resource_proxy[SUB_PROCESS_STATE].append(True)
File "<string>", line 2, in append
File "/usr/lib64/python3.11/multiprocessing/managers.py", line 818, in _callmethod
self._connect()
File "/usr/lib64/python3.11/multiprocessing/managers.py", line 805, in _connect
conn = self._Client(self._token.address, authkey=self._authkey)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/lib64/python3.11/multiprocessing/connection.py", line 518, in Client
c = SocketClient(address)
^^^^^^^^^^^^^^^^^^^^^
File "/usr/lib64/python3.11/multiprocessing/connection.py", line 646, in SocketClient
s.connect(address)
ConnectionRefusedError: [Errno 111] Connection refused
[W compiler_depend.ts:57] Warning: EZ9999: Inner Error!
EZ9999: [PID: 139] 2025-08-13-13:41:59.690.407 The error from device(0), serial number is 7, there is an error of aicore, core id is 7, error code = 0x800000, dump info: pc start: 0x8001240000000f0, current: 0x1240000001b8, vec error info: 0xf9f95f1, mte error info: 0x300006b, ifu error info: 0x23f438f54fe80, ccu error info: 0x5838e1f90004f4fd, cube error info: 0xe0, biu error info: 0, aic error mask: 0x65000200d00028c, para base: 0x12c000261000, errorStr: The DDR address of the MTE instruction is out of range.[FUNC:PrintCoreErrorInfo][FILE:device_error_core_proc.cc][LINE:46]
TraceBack (most recent call last):
The extend info from device(0), serial number is 7, there is aicore error, core id is 7, aicore int: 0x10, aicore error2: 0, axi clamp ctrl: 0, axi clamp state: 0x1717, biu status0: 0x101d14000000000, biu status1: 0x80000201020000, clk gate mask: 0, dbg addr: 0, ecc en: 0, mte ccu ecc 1bit error: 0x3b80000000000000, vector cube ecc 1bit error: 0, run stall: 0x1, dbg data0: 0, dbg data1: 0, dbg data2: 0, dbg data3: 0, dfx data: 0xe9[FUNC:PrintCoreErrorInfo][FILE:device_error_core_proc.cc][LINE:77]
The dha(mata) info from device(0), dha id is 0, dha status 1 info:0x23[FUNC:ProcessCoreErrorInfo][FILE:device_error_core_proc.cc][LINE:167]
The dha(mata) info from device(0), dha id is 1, dha status 1 info:0x3[FUNC:ProcessCoreErrorInfo][FILE:device_error_core_proc.cc][LINE:167]
The device(0), core list[0-0], error code is:[FUNC:PrintCoreInfoErrMsg][FILE:device_error_core_proc.cc][LINE:100]
coreId( 0): 0x800000 [FUNC:PrintCoreInfoErrMsg][FILE:device_error_core_proc.cc][LINE:114]
Kernel task happen error, retCode=0x26, [aicore exception].[FUNC:PreCheckTaskErr][FILE:davinci_kernel_task.cc][LINE:1539]
[AIC_INFO] after execute:args print end[FUNC:GetError][FILE:stream.cc][LINE:1183]
Aicore kernel execute failed, device_id=0, stream_id=1, report_stream_id=1, task_id=2, flip_num=0, fault kernel_name=ZerosLike_d2db1a80c523e7e59a032c95969880af_high_performance_210000000, fault kernel info ext=none, program id=0, hash=3724941331859449102.[FUNC:GetError][FILE:stream.cc][LINE:1183]
rtStreamSynchronize execute failed, reason=[aicore exception][FUNC:FuncErrorReason][FILE:error_message_manage.cc][LINE:53]
synchronize stream failed, runtime result = 507015[FUNC:ReportCallError][FILE:log_inner.cpp][LINE:161]
DEVICE[0] PID[139]:
EXCEPTION STREAM:
Exception info:TGID=23425, model id=65535, stream id=1, stream phase=SCHEDULE
Message info[0]:RTS_HWTS: Aicore exception, slot_id=2, stream_id=1
Other info[0]:time=2025-08-13-13:41:58.920.617, function=process_hwts_error_exception, line=646, error code=0x26 (function copy_between_host_and_device_opapi)
Process ForkServerProcess-5:
Traceback (most recent call last):
File "/usr/lib64/python3.11/multiprocessing/managers.py", line 814, in _callmethod
conn = self._tls.connection
^^^^^^^^^^^^^^^^^^^^
AttributeError: 'ForkAwareLocal' object has no attribute 'connection'
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/usr/lib64/python3.11/multiprocessing/process.py", line 314, in _bootstrap
self.run()
File "/usr/lib64/python3.11/multiprocessing/process.py", line 108, in run
self._target(*self._args, **self._kwargs)
File "/usr/local/Ascend/ascend-toolkit/latest/python/site-packages/tbe/common/repository_manager/route.py", line 71, in wrapper
raise exp
File "/usr/local/Ascend/ascend-toolkit/latest/python/site-packages/tbe/common/repository_manager/route.py", line 63, in wrapper
func(*args, **kwargs)
File "/usr/local/Ascend/ascend-toolkit/latest/python/site-packages/tbe/common/repository_manager/route.py", line 264, in task_distribute
resource_proxy[SUB_PROCESS_STATE].append(True)
File "<string>", line 2, in append
File "/usr/lib64/python3.11/multiprocessing/managers.py", line 818, in _callmethod
self._connect()
File "/usr/lib64/python3.11/multiprocessing/managers.py", line 805, in _connect
conn = self._Client(self._token.address, authkey=self._authkey)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/lib64/python3.11/multiprocessing/connection.py", line 518, in Client
c = SocketClient(address)
^^^^^^^^^^^^^^^^^^^^^
File "/usr/lib64/python3.11/multiprocessing/connection.py", line 646, in SocketClient
s.connect(address)
ConnectionRefusedError: [Errno 111] Connection refused
Process ForkServerProcess-7:
Traceback (most recent call last):
File "/usr/lib64/python3.11/multiprocessing/managers.py", line 814, in _callmethod
conn = self._tls.connection
^^^^^^^^^^^^^^^^^^^^
AttributeError: 'ForkAwareLocal' object has no attribute 'connection'
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/usr/lib64/python3.11/multiprocessing/process.py", line 314, in _bootstrap
self.run()
File "/usr/lib64/python3.11/multiprocessing/process.py", line 108, in run
self._target(*self._args, **self._kwargs)
File "/usr/local/Ascend/ascend-toolkit/latest/python/site-packages/tbe/common/repository_manager/route.py", line 71, in wrapper
raise exp
File "/usr/local/Ascend/ascend-toolkit/latest/python/site-packages/tbe/common/repository_manager/route.py", line 63, in wrapper
func(*args, **kwargs)
File "/usr/local/Ascend/ascend-toolkit/latest/python/site-packages/tbe/common/repository_manager/route.py", line 264, in task_distribute
resource_proxy[SUB_PROCESS_STATE].append(True)
File "<string>", line 2, in append
File "/usr/lib64/python3.11/multiprocessing/managers.py", line 818, in _callmethod
self._connect()
File "/usr/lib64/python3.11/multiprocessing/managers.py", line 805, in _connect
conn = self._Client(self._token.address, authkey=self._authkey)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/lib64/python3.11/multiprocessing/connection.py", line 518, in Client
c = SocketClient(address)
^^^^^^^^^^^^^^^^^^^^^
File "/usr/lib64/python3.11/multiprocessing/connection.py", line 646, in SocketClient
s.connect(address)
ConnectionRefusedError: [Errno 111] Connection refused
Process ForkServerProcess-9:
Traceback (most recent call last):
File "/usr/lib64/python3.11/multiprocessing/managers.py", line 814, in _callmethod
conn = self._tls.connection
^^^^^^^^^^^^^^^^^^^^
AttributeError: 'ForkAwareLocal' object has no attribute 'connection'
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/usr/lib64/python3.11/multiprocessing/process.py", line 314, in _bootstrap
self.run()
File "/usr/lib64/python3.11/multiprocessing/process.py", line 108, in run
self._target(*self._args, **self._kwargs)
File "/usr/local/Ascend/ascend-toolkit/latest/python/site-packages/tbe/common/repository_manager/route.py", line 71, in wrapper
raise exp
File "/usr/local/Ascend/ascend-toolkit/latest/python/site-packages/tbe/common/repository_manager/route.py", line 63, in wrapper
func(*args, **kwargs)
File "/usr/local/Ascend/ascend-toolkit/latest/python/site-packages/tbe/common/repository_manager/route.py", line 264, in task_distribute
resource_proxy[SUB_PROCESS_STATE].append(True)
File "<string>", line 2, in append
File "/usr/lib64/python3.11/multiprocessing/managers.py", line 818, in _callmethod
self._connect()
File "/usr/lib64/python3.11/multiprocessing/managers.py", line 805, in _connect
conn = self._Client(self._token.address, authkey=self._authkey)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/lib64/python3.11/multiprocessing/connection.py", line 518, in Client
c = SocketClient(address)
^^^^^^^^^^^^^^^^^^^^^
File "/usr/lib64/python3.11/multiprocessing/connection.py", line 646, in SocketClient
s.connect(address)
ConnectionRefusedError: [Errno 111] Connection refused
Process ForkServerProcess-6:
[ERROR] TBE Subprocess[task_distribute] raise error[], main process disappeared!
[ERROR] TBE Subprocess[task_distribute] raise error[], main process disappeared!
[ERROR] TBE Subprocess[task_distribute] raise error[], main process disappeared!
Traceback (most recent call last):
[ERROR] TBE Subprocess[task_distribute] raise error[], main process disappeared!
File "/usr/lib64/python3.11/multiprocessing/managers.py", line 814, in _callmethod
conn = self._tls.connection
^^^^^^^^^^^^^^^^^^^^
AttributeError: 'ForkAwareLocal' object has no attribute 'connection'
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
[ERROR] TBE Subprocess[task_distribute] raise error[], main process disappeared!
[ERROR] TBE Subprocess[task_distribute] raise error[], main process disappeared!
File "/usr/lib64/python3.11/multiprocessing/process.py", line 314, in _bootstrap
self.run()
File "/usr/lib64/python3.11/multiprocessing/process.py", line 108, in run
self._target(*self._args, **self._kwargs)
File "/usr/local/Ascend/ascend-toolkit/latest/python/site-packages/tbe/common/repository_manager/route.py", line 71, in wrapper
raise exp
File "/usr/local/Ascend/ascend-toolkit/latest/python/site-packages/tbe/common/repository_manager/route.py", line 63, in wrapper
func(*args, **kwargs)
File "/usr/local/Ascend/ascend-toolkit/latest/python/site-packages/tbe/common/repository_manager/route.py", line 264, in task_distribute
resource_proxy[SUB_PROCESS_STATE].append(True)
File "<string>", line 2, in append
File "/usr/lib64/python3.11/multiprocessing/managers.py", line 818, in _callmethod
self._connect()
File "/usr/lib64/python3.11/multiprocessing/managers.py", line 805, in _connect
conn = self._Client(self._token.address, authkey=self._authkey)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/lib64/python3.11/multiprocessing/connection.py", line 518, in Client
c = SocketClient(address)
^^^^^^^^^^^^^^^^^^^^^
File "/usr/lib64/python3.11/multiprocessing/connection.py", line 646, in SocketClient
s.connect(address)
ConnectionRefusedError: [Errno 111] Connection refused
Process ForkServerProcess-4:
Traceback (most recent call last):
File "/usr/lib64/python3.11/multiprocessing/managers.py", line 814, in _callmethod
conn = self._tls.connection
^^^^^^^^^^^^^^^^^^^^
AttributeError: 'ForkAwareLocal' object has no attribute 'connection'
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/usr/lib64/python3.11/multiprocessing/process.py", line 314, in _bootstrap
self.run()
File "/usr/lib64/python3.11/multiprocessing/process.py", line 108, in run
self._target(*self._args, **self._kwargs)
File "/usr/local/Ascend/ascend-toolkit/latest/python/site-packages/tbe/common/repository_manager/route.py", line 71, in wrapper
raise exp
File "/usr/local/Ascend/ascend-toolkit/latest/python/site-packages/tbe/common/repository_manager/route.py", line 63, in wrapper
func(*args, **kwargs)
File "/usr/local/Ascend/ascend-toolkit/latest/python/site-packages/tbe/common/repository_manager/route.py", line 264, in task_distribute
resource_proxy[SUB_PROCESS_STATE].append(True)
File "<string>", line 2, in append
File "/usr/lib64/python3.11/multiprocessing/managers.py", line 818, in _callmethod
self._connect()
File "/usr/lib64/python3.11/multiprocessing/managers.py", line 805, in _connect
conn = self._Client(self._token.address, authkey=self._authkey)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/lib64/python3.11/multiprocessing/connection.py", line 518, in Client
c = SocketClient(address)
^^^^^^^^^^^^^^^^^^^^^
File "/usr/lib64/python3.11/multiprocessing/connection.py", line 646, in SocketClient
s.connect(address)
ConnectionRefusedError: [Errno 111] Connection refused
Process ForkServerProcess-7:
Traceback (most recent call last):
File "/usr/lib64/python3.11/multiprocessing/managers.py", line 814, in _callmethod
conn = self._tls.connection
^^^^^^^^^^^^^^^^^^^^
AttributeError: 'ForkAwareLocal' object has no attribute 'connection'
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/usr/lib64/python3.11/multiprocessing/process.py", line 314, in _bootstrap
self.run()
File "/usr/lib64/python3.11/multiprocessing/process.py", line 108, in run
self._target(*self._args, **self._kwargs)
File "/usr/local/Ascend/ascend-toolkit/latest/python/site-packages/tbe/common/repository_manager/route.py", line 71, in wrapper
raise exp
File "/usr/local/Ascend/ascend-toolkit/latest/python/site-packages/tbe/common/repository_manager/route.py", line 63, in wrapper
func(*args, **kwargs)
File "/usr/local/Ascend/ascend-toolkit/latest/python/site-packages/tbe/common/repository_manager/route.py", line 264, in task_distribute
resource_proxy[SUB_PROCESS_STATE].append(True)
File "<string>", line 2, in append
File "/usr/lib64/python3.11/multiprocessing/managers.py", line 818, in _callmethod
self._connect()
File "/usr/lib64/python3.11/multiprocessing/managers.py", line 805, in _connect
conn = self._Client(self._token.address, authkey=self._authkey)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/lib64/python3.11/multiprocessing/connection.py", line 518, in Client
c = SocketClient(address)
^^^^^^^^^^^^^^^^^^^^^
File "/usr/lib64/python3.11/multiprocessing/connection.py", line 646, in SocketClient
s.connect(address)
ConnectionRefusedError: [Errno 111] Connection refused
Process ForkServerProcess-8:
Traceback (most recent call last):
File "/usr/lib64/python3.11/multiprocessing/managers.py", line 814, in _callmethod
conn = self._tls.connection
^^^^^^^^^^^^^^^^^^^^
AttributeError: 'ForkAwareLocal' object has no attribute 'connection'
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/usr/lib64/python3.11/multiprocessing/process.py", line 314, in _bootstrap
self.run()
File "/usr/lib64/python3.11/multiprocessing/process.py", line 108, in run
self._target(*self._args, **self._kwargs)
File "/usr/local/Ascend/ascend-toolkit/latest/python/site-packages/tbe/common/repository_manager/route.py", line 71, in wrapper
raise exp
File "/usr/local/Ascend/ascend-toolkit/latest/python/site-packages/tbe/common/repository_manager/route.py", line 63, in wrapper
func(*args, **kwargs)
File "/usr/local/Ascend/ascend-toolkit/latest/python/site-packages/tbe/common/repository_manager/route.py", line 264, in task_distribute
resource_proxy[SUB_PROCESS_STATE].append(True)
File "<string>", line 2, in append
File "/usr/lib64/python3.11/multiprocessing/managers.py", line 818, in _callmethod
self._connect()
File "/usr/lib64/python3.11/multiprocessing/managers.py", line 805, in _connect
conn = self._Client(self._token.address, authkey=self._authkey)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/lib64/python3.11/multiprocessing/connection.py", line 518, in Client
c = SocketClient(address)
^^^^^^^^^^^^^^^^^^^^^
File "/usr/lib64/python3.11/multiprocessing/connection.py", line 646, in SocketClient
s.connect(address)
ConnectionRefusedError: [Errno 111] Connection refused
/usr/lib64/python3.11/multiprocessing/resource_tracker.py:254: UserWarning: resource_tracker: There appear to be 30 leaked semaphore objects to clean up at shutdown
warnings.warn('resource_tracker: There appear to be %d '
/usr/lib64/python3.11/multiprocessing/resource_tracker.py:254: UserWarning: resource_tracker: There appear to be 30 leaked semaphore objects to clean up at shutdown
warnings.warn('resource_tracker: There appear to be %d '
Daemon is killing...
Killed
运行平台信息如下:
NAME="Kylin" VERSION="银河麒麟桌面操作系统V10 SP1" VERSION_US="Kylin Linux Desktop V10 SP1" ID=kylin ID_LIKE=debian PRETTY_NAME="Kylin V10 SP1" VERSION_ID="v10" HOME_URL="http://www.kylinos.cn/" SUPPORT_URL="http://www.kylinos.cn/support/technology.html" BUG_REPORT_URL="http://www.kylinos.cn/" PRIVACY_POLICY_URL="http://www.kylinos.cn" VERSION_CODENAME=kylin UBUNTU_CODENAME=kylin PROJECT_CODENAME=V10SP1 KYLIN_RELEASE_ID="2503"
推理卡:300iduo×1
驱动版本:Ascend-hdk-310p-npu-driver_25.2.0_linux-aarch64
固件版本:Ascend-hdk-310p-npu-firmware_7.7.0.6.236
mindie版本:2.1.RC1-300I-Duo-py311-openeuler24.03-lts
模型信息:DeepSeek-R1-Distill-Qwen-1.5B
问题描述:使用相同硬件平台在其他操作系统时可以正常推理模型,但是在”银河麒麟桌面操作系统V10 SP1“下运行报错如下:
[W compiler_depend.ts:57] Warning: EZ9999: Inner Error!
EZ9999: [PID: 141] 2025-08-13-13:41:58.752.109 The error from device(1), serial number is 6, there is an error of aicore, core id is 5, error code = 0x800000, dump info: pc start: 0x8001242000000f0, current: 0x1242000001b8, vec error info: 0x1fbefefd, mte error info: 0x300006b, ifu error info: 0x35e917b2f5b80, ccu error info: 0xbab19eed004f2047, cube error info: 0xbb, biu error info: 0, aic error mask: 0x65000200d00028c, para base: 0x12c000261000, errorStr: The DDR address of the MTE instruction is out of range.[FUNC:PrintCoreErrorInfo][FILE:device_error_core_proc.cc][LINE:46]
TraceBack (most recent call last):
The extend info from device(1), serial number is 6, there is aicore error, core id is 5, aicore int: 0x10, aicore error2: 0, axi clamp ctrl: 0, axi clamp state: 0x1717, biu status0: 0x101d14000000000, biu status1: 0x80000201020000, clk gate mask: 0, dbg addr: 0, ecc en: 0, mte ccu ecc 1bit error: 0x1700000000000000, vector cube ecc 1bit error: 0, run stall: 0x1, dbg data0: 0, dbg data1: 0, dbg data2: 0, dbg data3: 0, dfx data: 0xf6[FUNC:PrintCoreErrorInfo][FILE:device_error_core_proc.cc][LINE:77]
The dha(mata) info from device(1), dha id is 0, dha status 1 info:0x23[FUNC:ProcessCoreErrorInfo][FILE:device_error_core_proc.cc][LINE:167]
The dha(mata) info from device(1), dha id is 1, dha status 1 info:0x3[FUNC:ProcessCoreErrorInfo][FILE:device_error_core_proc.cc][LINE:167]
The device(1), core list[0-0], error code is:[FUNC:PrintCoreInfoErrMsg][FILE:device_error_core_proc.cc][LINE:100]
coreId( 0): 0x800000 [FUNC:PrintCoreInfoErrMsg][FILE:device_error_core_proc.cc][LINE:114]
Kernel task happen error, retCode=0x26, [aicore exception].[FUNC:PreCheckTaskErr][FILE:davinci_kernel_task.cc][LINE:1539]
[AIC_INFO] after execute:args print end[FUNC:GetError][FILE:stream.cc][LINE:1183]
Aicore kernel execute failed, device_id=1, stream_id=2, report_stream_id=2, task_id=2, flip_num=0, fault kernel_name=ZerosLike_d2db1a80c523e7e59a032c95969880af_high_performance_210000000, fault kernel info ext=none, program id=0, hash=3724941331859449102.[FUNC:GetError][FILE:stream.cc][LINE:1183]
rtStreamSynchronize execute failed, reason=[aicore exception][FUNC:FuncErrorReason][FILE:error_message_manage.cc][LINE:53]
synchronize stream failed, runtime result = 507015[FUNC:ReportCallError][FILE:log_inner.cpp][LINE:161]
DEVICE[1] PID[141]:
EXCEPTION STREAM:
Exception info:TGID=23427, model id=65535, stream id=2, stream phase=SCHEDULE
Message info[0]:RTS_HWTS: Aicore exception, slot_id=7, stream_id=2
Other info[0]:time=2025-08-13-13:41:58.064.062, function=process_hwts_error_exception, line=646, error code=0x26 (function copy_between_host_and_device_opapi)
[ERROR] TBE Subprocess[task_distribute] raise error[], main process disappeared!
Process ForkServerProcess-8:
Traceback (most recent call last):
File "/usr/lib64/python3.11/multiprocessing/managers.py", line 814, in _callmethod
conn = self._tls.connection
^^^^^^^^^^^^^^^^^^^^
AttributeError: 'ForkAwareLocal' object has no attribute 'connection'
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/usr/lib64/python3.11/multiprocessing/process.py", line 314, in _bootstrap
self.run()
File "/usr/lib64/python3.11/multiprocessing/process.py", line 108, in run
self._target(*self._args, **self._kwargs)
File "/usr/local/Ascend/ascend-toolkit/latest/python/site-packages/tbe/common/repository_manager/route.py", line 71, in wrapper
raise exp
File "/usr/local/Ascend/ascend-toolkit/latest/python/site-packages/tbe/common/repository_manager/route.py", line 63, in wrapper
func(*args, **kwargs)
File "/usr/local/Ascend/ascend-toolkit/latest/python/site-packages/tbe/common/repository_manager/route.py", line 264, in task_distribute
resource_proxy[SUB_PROCESS_STATE].append(True)
File "<string>", line 2, in append
File "/usr/lib64/python3.11/multiprocessing/managers.py", line 818, in _callmethod
self._connect()
File "/usr/lib64/python3.11/multiprocessing/managers.py", line 805, in _connect
conn = self._Client(self._token.address, authkey=self._authkey)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/lib64/python3.11/multiprocessing/connection.py", line 518, in Client
c = SocketClient(address)
^^^^^^^^^^^^^^^^^^^^^
File "/usr/lib64/python3.11/multiprocessing/connection.py", line 646, in SocketClient
s.connect(address)
ConnectionRefusedError: [Errno 111] Connection refused
Process ForkServerProcess-3:
Traceback (most recent call last):
File "/usr/lib64/python3.11/multiprocessing/managers.py", line 814, in _callmethod
conn = self._tls.connection
^^^^^^^^^^^^^^^^^^^^
AttributeError: 'ForkAwareLocal' object has no attribute 'connection'
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/usr/lib64/python3.11/multiprocessing/process.py", line 314, in _bootstrap
self.run()
File "/usr/lib64/python3.11/multiprocessing/process.py", line 108, in run
self._target(*self._args, **self._kwargs)
File "/usr/local/Ascend/ascend-toolkit/latest/python/site-packages/tbe/common/repository_manager/route.py", line 71, in wrapper
raise exp
File "/usr/local/Ascend/ascend-toolkit/latest/python/site-packages/tbe/common/repository_manager/route.py", line 63, in wrapper
func(*args, **kwargs)
File "/usr/local/Ascend/ascend-toolkit/latest/python/site-packages/tbe/common/repository_manager/route.py", line 264, in task_distribute
resource_proxy[SUB_PROCESS_STATE].append(True)
File "<string>", line 2, in append
File "/usr/lib64/python3.11/multiprocessing/managers.py", line 818, in _callmethod
self._connect()
File "/usr/lib64/python3.11/multiprocessing/managers.py", line 805, in _connect
conn = self._Client(self._token.address, authkey=self._authkey)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/lib64/python3.11/multiprocessing/connection.py", line 518, in Client
c = SocketClient(address)
^^^^^^^^^^^^^^^^^^^^^
File "/usr/lib64/python3.11/multiprocessing/connection.py", line 646, in SocketClient
s.connect(address)
ConnectionRefusedError: [Errno 111] Connection refused
[W compiler_depend.ts:57] Warning: EZ9999: Inner Error!
EZ9999: [PID: 139] 2025-08-13-13:41:59.690.407 The error from device(0), serial number is 7, there is an error of aicore, core id is 7, error code = 0x800000, dump info: pc start: 0x8001240000000f0, current: 0x1240000001b8, vec error info: 0xf9f95f1, mte error info: 0x300006b, ifu error info: 0x23f438f54fe80, ccu error info: 0x5838e1f90004f4fd, cube error info: 0xe0, biu error info: 0, aic error mask: 0x65000200d00028c, para base: 0x12c000261000, errorStr: The DDR address of the MTE instruction is out of range.[FUNC:PrintCoreErrorInfo][FILE:device_error_core_proc.cc][LINE:46]
TraceBack (most recent call last):
The extend info from device(0), serial number is 7, there is aicore error, core id is 7, aicore int: 0x10, aicore error2: 0, axi clamp ctrl: 0, axi clamp state: 0x1717, biu status0: 0x101d14000000000, biu status1: 0x80000201020000, clk gate mask: 0, dbg addr: 0, ecc en: 0, mte ccu ecc 1bit error: 0x3b80000000000000, vector cube ecc 1bit error: 0, run stall: 0x1, dbg data0: 0, dbg data1: 0, dbg data2: 0, dbg data3: 0, dfx data: 0xe9[FUNC:PrintCoreErrorInfo][FILE:device_error_core_proc.cc][LINE:77]
The dha(mata) info from device(0), dha id is 0, dha status 1 info:0x23[FUNC:ProcessCoreErrorInfo][FILE:device_error_core_proc.cc][LINE:167]
The dha(mata) info from device(0), dha id is 1, dha status 1 info:0x3[FUNC:ProcessCoreErrorInfo][FILE:device_error_core_proc.cc][LINE:167]
The device(0), core list[0-0], error code is:[FUNC:PrintCoreInfoErrMsg][FILE:device_error_core_proc.cc][LINE:100]
coreId( 0): 0x800000 [FUNC:PrintCoreInfoErrMsg][FILE:device_error_core_proc.cc][LINE:114]
Kernel task happen error, retCode=0x26, [aicore exception].[FUNC:PreCheckTaskErr][FILE:davinci_kernel_task.cc][LINE:1539]
[AIC_INFO] after execute:args print end[FUNC:GetError][FILE:stream.cc][LINE:1183]
Aicore kernel execute failed, device_id=0, stream_id=1, report_stream_id=1, task_id=2, flip_num=0, fault kernel_name=ZerosLike_d2db1a80c523e7e59a032c95969880af_high_performance_210000000, fault kernel info ext=none, program id=0, hash=3724941331859449102.[FUNC:GetError][FILE:stream.cc][LINE:1183]
rtStreamSynchronize execute failed, reason=[aicore exception][FUNC:FuncErrorReason][FILE:error_message_manage.cc][LINE:53]
synchronize stream failed, runtime result = 507015[FUNC:ReportCallError][FILE:log_inner.cpp][LINE:161]
DEVICE[0] PID[139]:
EXCEPTION STREAM:
Exception info:TGID=23425, model id=65535, stream id=1, stream phase=SCHEDULE
Message info[0]:RTS_HWTS: Aicore exception, slot_id=2, stream_id=1
Other info[0]:time=2025-08-13-13:41:58.920.617, function=process_hwts_error_exception, line=646, error code=0x26 (function copy_between_host_and_device_opapi)
Process ForkServerProcess-5:
Traceback (most recent call last):
File "/usr/lib64/python3.11/multiprocessing/managers.py", line 814, in _callmethod
conn = self._tls.connection
^^^^^^^^^^^^^^^^^^^^
AttributeError: 'ForkAwareLocal' object has no attribute 'connection'
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/usr/lib64/python3.11/multiprocessing/process.py", line 314, in _bootstrap
self.run()
File "/usr/lib64/python3.11/multiprocessing/process.py", line 108, in run
self._target(*self._args, **self._kwargs)
File "/usr/local/Ascend/ascend-toolkit/latest/python/site-packages/tbe/common/repository_manager/route.py", line 71, in wrapper
raise exp
File "/usr/local/Ascend/ascend-toolkit/latest/python/site-packages/tbe/common/repository_manager/route.py", line 63, in wrapper
func(*args, **kwargs)
File "/usr/local/Ascend/ascend-toolkit/latest/python/site-packages/tbe/common/repository_manager/route.py", line 264, in task_distribute
resource_proxy[SUB_PROCESS_STATE].append(True)
File "<string>", line 2, in append
File "/usr/lib64/python3.11/multiprocessing/managers.py", line 818, in _callmethod
self._connect()
File "/usr/lib64/python3.11/multiprocessing/managers.py", line 805, in _connect
conn = self._Client(self._token.address, authkey=self._authkey)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/lib64/python3.11/multiprocessing/connection.py", line 518, in Client
c = SocketClient(address)
^^^^^^^^^^^^^^^^^^^^^
File "/usr/lib64/python3.11/multiprocessing/connection.py", line 646, in SocketClient
s.connect(address)
ConnectionRefusedError: [Errno 111] Connection refused
Process ForkServerProcess-7:
Traceback (most recent call last):
File "/usr/lib64/python3.11/multiprocessing/managers.py", line 814, in _callmethod
conn = self._tls.connection
^^^^^^^^^^^^^^^^^^^^
AttributeError: 'ForkAwareLocal' object has no attribute 'connection'
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/usr/lib64/python3.11/multiprocessing/process.py", line 314, in _bootstrap
self.run()
File "/usr/lib64/python3.11/multiprocessing/process.py", line 108, in run
self._target(*self._args, **self._kwargs)
File "/usr/local/Ascend/ascend-toolkit/latest/python/site-packages/tbe/common/repository_manager/route.py", line 71, in wrapper
raise exp
File "/usr/local/Ascend/ascend-toolkit/latest/python/site-packages/tbe/common/repository_manager/route.py", line 63, in wrapper
func(*args, **kwargs)
File "/usr/local/Ascend/ascend-toolkit/latest/python/site-packages/tbe/common/repository_manager/route.py", line 264, in task_distribute
resource_proxy[SUB_PROCESS_STATE].append(True)
File "<string>", line 2, in append
File "/usr/lib64/python3.11/multiprocessing/managers.py", line 818, in _callmethod
self._connect()
File "/usr/lib64/python3.11/multiprocessing/managers.py", line 805, in _connect
conn = self._Client(self._token.address, authkey=self._authkey)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/lib64/python3.11/multiprocessing/connection.py", line 518, in Client
c = SocketClient(address)
^^^^^^^^^^^^^^^^^^^^^
File "/usr/lib64/python3.11/multiprocessing/connection.py", line 646, in SocketClient
s.connect(address)
ConnectionRefusedError: [Errno 111] Connection refused
Process ForkServerProcess-9:
Traceback (most recent call last):
File "/usr/lib64/python3.11/multiprocessing/managers.py", line 814, in _callmethod
conn = self._tls.connection
^^^^^^^^^^^^^^^^^^^^
AttributeError: 'ForkAwareLocal' object has no attribute 'connection'
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/usr/lib64/python3.11/multiprocessing/process.py", line 314, in _bootstrap
self.run()
File "/usr/lib64/python3.11/multiprocessing/process.py", line 108, in run
self._target(*self._args, **self._kwargs)
File "/usr/local/Ascend/ascend-toolkit/latest/python/site-packages/tbe/common/repository_manager/route.py", line 71, in wrapper
raise exp
File "/usr/local/Ascend/ascend-toolkit/latest/python/site-packages/tbe/common/repository_manager/route.py", line 63, in wrapper
func(*args, **kwargs)
File "/usr/local/Ascend/ascend-toolkit/latest/python/site-packages/tbe/common/repository_manager/route.py", line 264, in task_distribute
resource_proxy[SUB_PROCESS_STATE].append(True)
File "<string>", line 2, in append
File "/usr/lib64/python3.11/multiprocessing/managers.py", line 818, in _callmethod
self._connect()
File "/usr/lib64/python3.11/multiprocessing/managers.py", line 805, in _connect
conn = self._Client(self._token.address, authkey=self._authkey)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/lib64/python3.11/multiprocessing/connection.py", line 518, in Client
c = SocketClient(address)
^^^^^^^^^^^^^^^^^^^^^
File "/usr/lib64/python3.11/multiprocessing/connection.py", line 646, in SocketClient
s.connect(address)
ConnectionRefusedError: [Errno 111] Connection refused
Process ForkServerProcess-6:
[ERROR] TBE Subprocess[task_distribute] raise error[], main process disappeared!
[ERROR] TBE Subprocess[task_distribute] raise error[], main process disappeared!
[ERROR] TBE Subprocess[task_distribute] raise error[], main process disappeared!
Traceback (most recent call last):
[ERROR] TBE Subprocess[task_distribute] raise error[], main process disappeared!
File "/usr/lib64/python3.11/multiprocessing/managers.py", line 814, in _callmethod
conn = self._tls.connection
^^^^^^^^^^^^^^^^^^^^
AttributeError: 'ForkAwareLocal' object has no attribute 'connection'
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
[ERROR] TBE Subprocess[task_distribute] raise error[], main process disappeared!
[ERROR] TBE Subprocess[task_distribute] raise error[], main process disappeared!
File "/usr/lib64/python3.11/multiprocessing/process.py", line 314, in _bootstrap
self.run()
File "/usr/lib64/python3.11/multiprocessing/process.py", line 108, in run
self._target(*self._args, **self._kwargs)
File "/usr/local/Ascend/ascend-toolkit/latest/python/site-packages/tbe/common/repository_manager/route.py", line 71, in wrapper
raise exp
File "/usr/local/Ascend/ascend-toolkit/latest/python/site-packages/tbe/common/repository_manager/route.py", line 63, in wrapper
func(*args, **kwargs)
File "/usr/local/Ascend/ascend-toolkit/latest/python/site-packages/tbe/common/repository_manager/route.py", line 264, in task_distribute
resource_proxy[SUB_PROCESS_STATE].append(True)
File "<string>", line 2, in append
File "/usr/lib64/python3.11/multiprocessing/managers.py", line 818, in _callmethod
self._connect()
File "/usr/lib64/python3.11/multiprocessing/managers.py", line 805, in _connect
conn = self._Client(self._token.address, authkey=self._authkey)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/lib64/python3.11/multiprocessing/connection.py", line 518, in Client
c = SocketClient(address)
^^^^^^^^^^^^^^^^^^^^^
File "/usr/lib64/python3.11/multiprocessing/connection.py", line 646, in SocketClient
s.connect(address)
ConnectionRefusedError: [Errno 111] Connection refused
Process ForkServerProcess-4:
Traceback (most recent call last):
File "/usr/lib64/python3.11/multiprocessing/managers.py", line 814, in _callmethod
conn = self._tls.connection
^^^^^^^^^^^^^^^^^^^^
AttributeError: 'ForkAwareLocal' object has no attribute 'connection'
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/usr/lib64/python3.11/multiprocessing/process.py", line 314, in _bootstrap
self.run()
File "/usr/lib64/python3.11/multiprocessing/process.py", line 108, in run
self._target(*self._args, **self._kwargs)
File "/usr/local/Ascend/ascend-toolkit/latest/python/site-packages/tbe/common/repository_manager/route.py", line 71, in wrapper
raise exp
File "/usr/local/Ascend/ascend-toolkit/latest/python/site-packages/tbe/common/repository_manager/route.py", line 63, in wrapper
func(*args, **kwargs)
File "/usr/local/Ascend/ascend-toolkit/latest/python/site-packages/tbe/common/repository_manager/route.py", line 264, in task_distribute
resource_proxy[SUB_PROCESS_STATE].append(True)
File "<string>", line 2, in append
File "/usr/lib64/python3.11/multiprocessing/managers.py", line 818, in _callmethod
self._connect()
File "/usr/lib64/python3.11/multiprocessing/managers.py", line 805, in _connect
conn = self._Client(self._token.address, authkey=self._authkey)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/lib64/python3.11/multiprocessing/connection.py", line 518, in Client
c = SocketClient(address)
^^^^^^^^^^^^^^^^^^^^^
File "/usr/lib64/python3.11/multiprocessing/connection.py", line 646, in SocketClient
s.connect(address)
ConnectionRefusedError: [Errno 111] Connection refused
Process ForkServerProcess-7:
Traceback (most recent call last):
File "/usr/lib64/python3.11/multiprocessing/managers.py", line 814, in _callmethod
conn = self._tls.connection
^^^^^^^^^^^^^^^^^^^^
AttributeError: 'ForkAwareLocal' object has no attribute 'connection'
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/usr/lib64/python3.11/multiprocessing/process.py", line 314, in _bootstrap
self.run()
File "/usr/lib64/python3.11/multiprocessing/process.py", line 108, in run
self._target(*self._args, **self._kwargs)
File "/usr/local/Ascend/ascend-toolkit/latest/python/site-packages/tbe/common/repository_manager/route.py", line 71, in wrapper
raise exp
File "/usr/local/Ascend/ascend-toolkit/latest/python/site-packages/tbe/common/repository_manager/route.py", line 63, in wrapper
func(*args, **kwargs)
File "/usr/local/Ascend/ascend-toolkit/latest/python/site-packages/tbe/common/repository_manager/route.py", line 264, in task_distribute
resource_proxy[SUB_PROCESS_STATE].append(True)
File "<string>", line 2, in append
File "/usr/lib64/python3.11/multiprocessing/managers.py", line 818, in _callmethod
self._connect()
File "/usr/lib64/python3.11/multiprocessing/managers.py", line 805, in _connect
conn = self._Client(self._token.address, authkey=self._authkey)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/lib64/python3.11/multiprocessing/connection.py", line 518, in Client
c = SocketClient(address)
^^^^^^^^^^^^^^^^^^^^^
File "/usr/lib64/python3.11/multiprocessing/connection.py", line 646, in SocketClient
s.connect(address)
ConnectionRefusedError: [Errno 111] Connection refused
Process ForkServerProcess-8:
Traceback (most recent call last):
File "/usr/lib64/python3.11/multiprocessing/managers.py", line 814, in _callmethod
conn = self._tls.connection
^^^^^^^^^^^^^^^^^^^^
AttributeError: 'ForkAwareLocal' object has no attribute 'connection'
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/usr/lib64/python3.11/multiprocessing/process.py", line 314, in _bootstrap
self.run()
File "/usr/lib64/python3.11/multiprocessing/process.py", line 108, in run
self._target(*self._args, **self._kwargs)
File "/usr/local/Ascend/ascend-toolkit/latest/python/site-packages/tbe/common/repository_manager/route.py", line 71, in wrapper
raise exp
File "/usr/local/Ascend/ascend-toolkit/latest/python/site-packages/tbe/common/repository_manager/route.py", line 63, in wrapper
func(*args, **kwargs)
File "/usr/local/Ascend/ascend-toolkit/latest/python/site-packages/tbe/common/repository_manager/route.py", line 264, in task_distribute
resource_proxy[SUB_PROCESS_STATE].append(True)
File "<string>", line 2, in append
File "/usr/lib64/python3.11/multiprocessing/managers.py", line 818, in _callmethod
self._connect()
File "/usr/lib64/python3.11/multiprocessing/managers.py", line 805, in _connect
conn = self._Client(self._token.address, authkey=self._authkey)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/lib64/python3.11/multiprocessing/connection.py", line 518, in Client
c = SocketClient(address)
^^^^^^^^^^^^^^^^^^^^^
File "/usr/lib64/python3.11/multiprocessing/connection.py", line 646, in SocketClient
s.connect(address)
ConnectionRefusedError: [Errno 111] Connection refused
/usr/lib64/python3.11/multiprocessing/resource_tracker.py:254: UserWarning: resource_tracker: There appear to be 30 leaked semaphore objects to clean up at shutdown
warnings.warn('resource_tracker: There appear to be %d '
/usr/lib64/python3.11/multiprocessing/resource_tracker.py:254: UserWarning: resource_tracker: There appear to be 30 leaked semaphore objects to clean up at shutdown
warnings.warn('resource_tracker: There appear to be %d '
Daemon is killing...
Killed