环境
atlas 300I DUO训练
使用docker方式启动服务化推理
系统：openeurle22.04
docker镜像：
镜像文件名称：RC2-300I-DUO-py311-openeuler24.03-lts.tar
使用docker的方式启动服务化推理出现如下错误：
# ./mindieservice_daemon
The old environment variable ATB_LOG_TO_STDOUT will be deprecated on 2025/12/31. Please use the new environment variable MINDIE_LOG_TO_STDOUT as soon as possible.
The old environment variable ATB_LOG_LEVEL will be deprecated on 2025/12/31. Please use the new environment variable MINDIE_LOG_LEVEL as soon as possible.
The old environment variable MINDIE_LLM_LOG_LEVEL will be deprecated on 2025/12/31. Please use the new environment variable MINDIE_LOG_LEVEL as soon as possible.
The old environment variable MINDIE_LLM_PYTHON_LOG_TO_STDOUT will be deprecated on 2025/12/31. Please use the new environment variable MINDIE_LOG_TO_STDOUT as soon as possible.
The old environment variable MINDIE_LLM_LOG_TO_STDOUT will be deprecated on 2025/12/31. Please use the new environment variable MINDIE_LOG_TO_STDOUT as soon as possible.
The old environment variable MINDIE_LLM_PYTHON_LOG_PATH will be deprecated on 2025/12/31. Please use the new environment variable MINDIE_LOG_PATH as soon as possible.
The old environment variable ATB_LOG_TO_FILE will be deprecated on 2025/12/31. Please use the new environment variable MINDIE_LOG_TO_FILE as soon as possible.
The old environment variable MINDIE_LLM_PYTHON_LOG_LEVEL will be deprecated on 2025/12/31. Please use the new environment variable MINDIE_LOG_LEVEL as soon as possible.
The old environment variable MINDIE_LLM_PYTHON_LOG_TO_FILE will be deprecated on 2025/12/31. Please use the new environment variable MINDIE_LOG_TO_FILE as soon as possible.
The old environment variable MINDIE_LLM_LOG_TO_FILE will be deprecated on 2025/12/31. Please use the new environment variable MINDIE_LOG_TO_FILE as soon as possible.
The old environment variable OCK_LOG_LEVEL will be deprecated on 2025/12/31. Please use the new environment variable MINDIE_LOG_LEVEL as soon as possible.
The old environment variable OCK_LOG_TO_STDOUT will be deprecated on 2025/12/31. Please use the new environment variable MINDIE_LOG_TO_STDOUT as soon as possible.
Log default log dir is ~/mindie/log, your can use env MINDIE_LOG_PATH to change log saving dir.
Log default log dir is ~/mindie/log, your can use env MINDIE_LOG_PATH to change log saving dir.
[msservice_profiler] [PID:2106] [DEBUG] [ReadEnable:344] profile enable_: false
[msservice_profiler] [PID:2106] [DEBUG] [ReadAclTaskTime:372] profile enableAclTaskTime_: false
The old environment variable ATB_LOG_TO_STDOUT will be deprecated on 2025/12/31. Please use the new environment variable MINDIE_LOG_TO_STDOUT as soon as possible.
The old environment variable ATB_LOG_LEVEL will be deprecated on 2025/12/31. Please use the new environment variable MINDIE_LOG_LEVEL as soon as possible.
The old environment variable MINDIE_LLM_LOG_LEVEL will be deprecated on 2025/12/31. Please use the new environment variable MINDIE_LOG_LEVEL as soon as possible.
The old environment variable MINDIE_LLM_PYTHON_LOG_TO_STDOUT will be deprecated on 2025/12/31. Please use the new environment variable MINDIE_LOG_TO_STDOUT as soon as possible.
The old environment variable MINDIE_LLM_LOG_TO_STDOUT will be deprecated on 2025/12/31. Please use the new environment variable MINDIE_LOG_TO_STDOUT as soon as possible.
The old environment variable MINDIE_LLM_PYTHON_LOG_PATH will be deprecated on 2025/12/31. Please use the new environment variable MINDIE_LOG_PATH as soon as possible.
The old environment variable ATB_LOG_TO_FILE will be deprecated on 2025/12/31. Please use the new environment variable MINDIE_LOG_TO_FILE as soon as possible.
The old environment variable MINDIE_LLM_PYTHON_LOG_LEVEL will be deprecated on 2025/12/31. Please use the new environment variable MINDIE_LOG_LEVEL as soon as possible.
The old environment variable MINDIE_LLM_PYTHON_LOG_TO_FILE will be deprecated on 2025/12/31. Please use the new environment variable MINDIE_LOG_TO_FILE as soon as possible.
The old environment variable MINDIE_LLM_LOG_TO_FILE will be deprecated on 2025/12/31. Please use the new environment variable MINDIE_LOG_TO_FILE as soon as possible.
The old environment variable OCK_LOG_LEVEL will be deprecated on 2025/12/31. Please use the new environment variable MINDIE_LOG_LEVEL as soon as possible.
The old environment variable OCK_LOG_TO_STDOUT will be deprecated on 2025/12/31. Please use the new environment variable MINDIE_LOG_TO_STDOUT as soon as possible.
Process ForkServerProcess-5:
Process ForkServerProcess-9:
Process ForkServerProcess-3:
Process ForkServerProcess-4:
Process ForkServerProcess-7:
Process ForkServerProcess-2:
Process ForkServerProcess-8:
Process ForkServerProcess-6:
Traceback (most recent call last):
Traceback (most recent call last):
Traceback (most recent call last):
Traceback (most recent call last):
Traceback (most recent call last):
Traceback (most recent call last):
Traceback (most recent call last):
Traceback (most recent call last):
File "/usr/lib64/python3.11/multiprocessing/managers.py", line 814, in _callmethod
conn = self._tls.connection
^^^^^^^^^^^^^^^^^^^^
File "/usr/lib64/python3.11/multiprocessing/managers.py", line 814, in _callmethod
conn = self._tls.connection
^^^^^^^^^^^^^^^^^^^^
AttributeError: 'ForkAwareLocal' object has no attribute 'connection'
AttributeError: 'ForkAwareLocal' object has no attribute 'connection'
File "/usr/lib64/python3.11/multiprocessing/managers.py", line 814, in _callmethod
conn = self._tls.connection
^^^^^^^^^^^^^^^^^^^^
File "/usr/lib64/python3.11/multiprocessing/managers.py", line 814, in _callmethod
conn = self._tls.connection
^^^^^^^^^^^^^^^^^^^^
File "/usr/lib64/python3.11/multiprocessing/managers.py", line 814, in _callmethod
conn = self._tls.connection
^^^^^^^^^^^^^^^^^^^^
File "/usr/lib64/python3.11/multiprocessing/managers.py", line 814, in _callmethod
conn = self._tls.connection
^^^^^^^^^^^^^^^^^^^^
AttributeError: 'ForkAwareLocal' object has no attribute 'connection'
AttributeError: 'ForkAwareLocal' object has no attribute 'connection'
AttributeError: 'ForkAwareLocal' object has no attribute 'connection'
File "/usr/lib64/python3.11/multiprocessing/managers.py", line 814, in _callmethod
conn = self._tls.connection
^^^^^^^^^^^^^^^^^^^^
During handling of the above exception, another exception occurred:
AttributeError: 'ForkAwareLocal' object has no attribute 'connection'
File "/usr/lib64/python3.11/multiprocessing/managers.py", line 814, in _callmethod
conn = self._tls.connection
^^^^^^^^^^^^^^^^^^^^
Traceback (most recent call last):
AttributeError: 'ForkAwareLocal' object has no attribute 'connection'
During handling of the above exception, another exception occurred:
AttributeError: 'ForkAwareLocal' object has no attribute 'connection'
During handling of the above exception, another exception occurred:
During handling of the above exception, another exception occurred:
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
Traceback (most recent call last):
Traceback (most recent call last):
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
Traceback (most recent call last):
File "/usr/lib64/python3.11/multiprocessing/process.py", line 314, in _bootstrap
self.run()
File "/usr/lib64/python3.11/multiprocessing/process.py", line 108, in run
self._target(*self._args, **self._kwargs)
File "/usr/local/Ascend/ascend-toolkit/latest/python/site-packages/tbe/common/repository_manager/route.py", line 71, in wrapper
raise exp
File "/usr/lib64/python3.11/multiprocessing/process.py", line 314, in _bootstrap
self.run()
File "/usr/lib64/python3.11/multiprocessing/process.py", line 314, in _bootstrap
self.run()
File "/usr/lib64/python3.11/multiprocessing/process.py", line 314, in _bootstrap
self.run()
File "/usr/lib64/python3.11/multiprocessing/process.py", line 314, in _bootstrap
self.run()
File "/usr/local/Ascend/ascend-toolkit/latest/python/site-packages/tbe/common/repository_manager/route.py", line 63, in wrapper
func(*args, **kwargs)
File "/usr/lib64/python3.11/multiprocessing/process.py", line 108, in run
self._target(*self._args, **self._kwargs)
File "/usr/lib64/python3.11/multiprocessing/process.py", line 108, in run
self._target(*self._args, **self._kwargs)
File "/usr/lib64/python3.11/multiprocessing/process.py", line 108, in run
self._target(*self._args, **self._kwargs)
File "/usr/lib64/python3.11/multiprocessing/process.py", line 314, in _bootstrap
self.run()
File "/usr/lib64/python3.11/multiprocessing/process.py", line 314, in _bootstrap
self.run()
File "/usr/local/Ascend/ascend-toolkit/latest/python/site-packages/tbe/common/repository_manager/route.py", line 264, in task_distribute
resource_proxy[SUB_PROCESS_STATE].append(True)
File "/usr/lib64/python3.11/multiprocessing/process.py", line 108, in run
self._target(*self._args, **self._kwargs)
File "/usr/local/Ascend/ascend-toolkit/latest/python/site-packages/tbe/common/repository_manager/route.py", line 71, in wrapper
raise exp
File "/usr/lib64/python3.11/multiprocessing/process.py", line 314, in _bootstrap
self.run()
File "/usr/local/Ascend/ascend-toolkit/latest/python/site-packages/tbe/common/repository_manager/route.py", line 71, in wrapper
raise exp
File "/usr/local/Ascend/ascend-toolkit/latest/python/site-packages/tbe/common/repository_manager/route.py", line 71, in wrapper
raise exp
File "/usr/lib64/python3.11/multiprocessing/process.py", line 108, in run
self._target(*self._args, **self._kwargs)
File "<string>", line 2, in append
File "/usr/lib64/python3.11/multiprocessing/process.py", line 108, in run
self._target(*self._args, **self._kwargs)
File "/usr/local/Ascend/ascend-toolkit/latest/python/site-packages/tbe/common/repository_manager/route.py", line 71, in wrapper
raise exp
File "/usr/local/Ascend/ascend-toolkit/latest/python/site-packages/tbe/common/repository_manager/route.py", line 63, in wrapper
func(*args, **kwargs)
File "/usr/local/Ascend/ascend-toolkit/latest/python/site-packages/tbe/common/repository_manager/route.py", line 63, in wrapper
func(*args, **kwargs)
File "/usr/local/Ascend/ascend-toolkit/latest/python/site-packages/tbe/common/repository_manager/route.py", line 63, in wrapper
func(*args, **kwargs)
File "/usr/lib64/python3.11/multiprocessing/process.py", line 108, in run
self._target(*self._args, **self._kwargs)
File "/usr/local/Ascend/ascend-toolkit/latest/python/site-packages/tbe/common/repository_manager/route.py", line 71, in wrapper
raise exp
File "/usr/lib64/python3.11/multiprocessing/managers.py", line 818, in _callmethod
self._connect()
File "/usr/local/Ascend/ascend-toolkit/latest/python/site-packages/tbe/common/repository_manager/route.py", line 71, in wrapper
raise exp
File "/usr/local/Ascend/ascend-toolkit/latest/python/site-packages/tbe/common/repository_manager/route.py", line 63, in wrapper
func(*args, **kwargs)
File "/usr/local/Ascend/ascend-toolkit/latest/python/site-packages/tbe/common/repository_manager/route.py", line 264, in task_distribute
resource_proxy[SUB_PROCESS_STATE].append(True)
File "/usr/local/Ascend/ascend-toolkit/latest/python/site-packages/tbe/common/repository_manager/route.py", line 264, in task_distribute
resource_proxy[SUB_PROCESS_STATE].append(True)
File "/usr/local/Ascend/ascend-toolkit/latest/python/site-packages/tbe/common/repository_manager/route.py", line 264, in task_distribute
resource_proxy[SUB_PROCESS_STATE].append(True)
File "/usr/local/Ascend/ascend-toolkit/latest/python/site-packages/tbe/common/repository_manager/route.py", line 71, in wrapper
raise exp
File "/usr/local/Ascend/ascend-toolkit/latest/python/site-packages/tbe/common/repository_manager/route.py", line 63, in wrapper
func(*args, **kwargs)
File "/usr/lib64/python3.11/multiprocessing/managers.py", line 805, in _connect
conn = self._Client(self._token.address, authkey=self._authkey)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/Ascend/ascend-toolkit/latest/python/site-packages/tbe/common/repository_manager/route.py", line 63, in wrapper
func(*args, **kwargs)
File "/usr/local/Ascend/ascend-toolkit/latest/python/site-packages/tbe/common/repository_manager/route.py", line 264, in task_distribute
resource_proxy[SUB_PROCESS_STATE].append(True)
File "<string>", line 2, in append
File "<string>", line 2, in append
File "<string>", line 2, in append
File "/usr/local/Ascend/ascend-toolkit/latest/python/site-packages/tbe/common/repository_manager/route.py", line 63, in wrapper
func(*args, **kwargs)
File "/usr/local/Ascend/ascend-toolkit/latest/python/site-packages/tbe/common/repository_manager/route.py", line 264, in task_distribute
resource_proxy[SUB_PROCESS_STATE].append(True)
File "/usr/lib64/python3.11/multiprocessing/connection.py", line 518, in Client
c = SocketClient(address)
^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/Ascend/ascend-toolkit/latest/python/site-packages/tbe/common/repository_manager/route.py", line 264, in task_distribute
resource_proxy[SUB_PROCESS_STATE].append(True)
File "/usr/lib64/python3.11/multiprocessing/managers.py", line 818, in _callmethod
self._connect()
File "/usr/lib64/python3.11/multiprocessing/managers.py", line 818, in _callmethod
self._connect()
File "/usr/lib64/python3.11/multiprocessing/managers.py", line 818, in _callmethod
self._connect()
File "<string>", line 2, in append
File "/usr/local/Ascend/ascend-toolkit/latest/python/site-packages/tbe/common/repository_manager/route.py", line 264, in task_distribute
resource_proxy[SUB_PROCESS_STATE].append(True)
File "<string>", line 2, in append
File "/usr/lib64/python3.11/multiprocessing/connection.py", line 646, in SocketClient
s.connect(address)
File "<string>", line 2, in append
File "/usr/lib64/python3.11/multiprocessing/managers.py", line 805, in _connect
conn = self._Client(self._token.address, authkey=self._authkey)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/lib64/python3.11/multiprocessing/managers.py", line 805, in _connect
conn = self._Client(self._token.address, authkey=self._authkey)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/lib64/python3.11/multiprocessing/managers.py", line 805, in _connect
conn = self._Client(self._token.address, authkey=self._authkey)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/lib64/python3.11/multiprocessing/connection.py", line 518, in Client
c = SocketClient(address)
^^^^^^^^^^^^^^^^^^^^^
File "/usr/lib64/python3.11/multiprocessing/managers.py", line 818, in _callmethod
self._connect()
File "<string>", line 2, in append
File "/usr/lib64/python3.11/multiprocessing/managers.py", line 818, in _callmethod
self._connect()
File "/usr/lib64/python3.11/multiprocessing/managers.py", line 818, in _callmethod
self._connect()
ConnectionRefusedError: [Errno 111] Connection refused
File "/usr/lib64/python3.11/multiprocessing/connection.py", line 518, in Client
c = SocketClient(address)
^^^^^^^^^^^^^^^^^^^^^
File "/usr/lib64/python3.11/multiprocessing/connection.py", line 518, in Client
c = SocketClient(address)
^^^^^^^^^^^^^^^^^^^^^
File "/usr/lib64/python3.11/multiprocessing/connection.py", line 646, in SocketClient
s.connect(address)
File "/usr/lib64/python3.11/multiprocessing/managers.py", line 805, in _connect
conn = self._Client(self._token.address, authkey=self._authkey)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/lib64/python3.11/multiprocessing/managers.py", line 818, in _callmethod
self._connect()
File "/usr/lib64/python3.11/multiprocessing/managers.py", line 805, in _connect
conn = self._Client(self._token.address, authkey=self._authkey)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/lib64/python3.11/multiprocessing/managers.py", line 805, in _connect
conn = self._Client(self._token.address, authkey=self._authkey)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/lib64/python3.11/multiprocessing/connection.py", line 646, in SocketClient
s.connect(address)
File "/usr/lib64/python3.11/multiprocessing/connection.py", line 646, in SocketClient
s.connect(address)
ConnectionRefusedError: [Errno 111] Connection refused
File "/usr/lib64/python3.11/multiprocessing/connection.py", line 518, in Client
c = SocketClient(address)
^^^^^^^^^^^^^^^^^^^^^
File "/usr/lib64/python3.11/multiprocessing/managers.py", line 805, in _connect
conn = self._Client(self._token.address, authkey=self._authkey)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/lib64/python3.11/multiprocessing/connection.py", line 518, in Client
c = SocketClient(address)
^^^^^^^^^^^^^^^^^^^^^
File "/usr/lib64/python3.11/multiprocessing/connection.py", line 518, in Client
c = SocketClient(address)
^^^^^^^^^^^^^^^^^^^^^
ConnectionRefusedError: [Errno 111] Connection refused
ConnectionRefusedError: [Errno 111] Connection refused
File "/usr/lib64/python3.11/multiprocessing/connection.py", line 646, in SocketClient
s.connect(address)
File "/usr/lib64/python3.11/multiprocessing/connection.py", line 518, in Client
c = SocketClient(address)
^^^^^^^^^^^^^^^^^^^^^
File "/usr/lib64/python3.11/multiprocessing/connection.py", line 646, in SocketClient
s.connect(address)
File "/usr/lib64/python3.11/multiprocessing/connection.py", line 646, in SocketClient
s.connect(address)
File "/usr/lib64/python3.11/multiprocessing/connection.py", line 646, in SocketClient
s.connect(address)
ConnectionRefusedError: [Errno 111] Connection refused
ConnectionRefusedError: [Errno 111] Connection refused
ConnectionRefusedError: [Errno 111] Connection refused
ConnectionRefusedError: [Errno 111] Connection refused
/usr/lib64/python3.11/multiprocessing/resource_tracker.py:254: UserWarning: resource_tracker: There appear to be 30 leaked semaphore objects to clean up at shutdown
warnings.warn('resource_tracker: There appear to be %d '
Daemon is killing...
Killed
点赞
收藏
回复
分享
举报
已解决
环境
atlas 300I DUO训练
使用docker方式启动服务化推理
系统：openeurle22.04
docker镜像：
镜像文件名称：RC2-300I-DUO-py311-openeuler24.03-lts.tar
使用docker的方式启动服务化推理出现如下错误：
# ./mindieservice_daemon
The old environment variable ATB_LOG_TO_STDOUT will be deprecated on 2025/12/31. Please use the new environment variable MINDIE_LOG_TO_STDOUT as soon as possible.
The old environment variable ATB_LOG_LEVEL will be deprecated on 2025/12/31. Please use the new environment variable MINDIE_LOG_LEVEL as soon as possible.
The old environment variable MINDIE_LLM_LOG_LEVEL will be deprecated on 2025/12/31. Please use the new environment variable MINDIE_LOG_LEVEL as soon as possible.
The old environment variable MINDIE_LLM_PYTHON_LOG_TO_STDOUT will be deprecated on 2025/12/31. Please use the new environment variable MINDIE_LOG_TO_STDOUT as soon as possible.
The old environment variable MINDIE_LLM_LOG_TO_STDOUT will be deprecated on 2025/12/31. Please use the new environment variable MINDIE_LOG_TO_STDOUT as soon as possible.
The old environment variable MINDIE_LLM_PYTHON_LOG_PATH will be deprecated on 2025/12/31. Please use the new environment variable MINDIE_LOG_PATH as soon as possible.
The old environment variable ATB_LOG_TO_FILE will be deprecated on 2025/12/31. Please use the new environment variable MINDIE_LOG_TO_FILE as soon as possible.
The old environment variable MINDIE_LLM_PYTHON_LOG_LEVEL will be deprecated on 2025/12/31. Please use the new environment variable MINDIE_LOG_LEVEL as soon as possible.
The old environment variable MINDIE_LLM_PYTHON_LOG_TO_FILE will be deprecated on 2025/12/31. Please use the new environment variable MINDIE_LOG_TO_FILE as soon as possible.
The old environment variable MINDIE_LLM_LOG_TO_FILE will be deprecated on 2025/12/31. Please use the new environment variable MINDIE_LOG_TO_FILE as soon as possible.
The old environment variable OCK_LOG_LEVEL will be deprecated on 2025/12/31. Please use the new environment variable MINDIE_LOG_LEVEL as soon as possible.
The old environment variable OCK_LOG_TO_STDOUT will be deprecated on 2025/12/31. Please use the new environment variable MINDIE_LOG_TO_STDOUT as soon as possible.
Log default log dir is ~/mindie/log, your can use env MINDIE_LOG_PATH to change log saving dir.
Log default log dir is ~/mindie/log, your can use env MINDIE_LOG_PATH to change log saving dir.
[msservice_profiler] [PID:2106] [DEBUG] [ReadEnable:344] profile enable_: false
[msservice_profiler] [PID:2106] [DEBUG] [ReadAclTaskTime:372] profile enableAclTaskTime_: false
The old environment variable ATB_LOG_TO_STDOUT will be deprecated on 2025/12/31. Please use the new environment variable MINDIE_LOG_TO_STDOUT as soon as possible.
The old environment variable ATB_LOG_LEVEL will be deprecated on 2025/12/31. Please use the new environment variable MINDIE_LOG_LEVEL as soon as possible.
The old environment variable MINDIE_LLM_LOG_LEVEL will be deprecated on 2025/12/31. Please use the new environment variable MINDIE_LOG_LEVEL as soon as possible.
The old environment variable MINDIE_LLM_PYTHON_LOG_TO_STDOUT will be deprecated on 2025/12/31. Please use the new environment variable MINDIE_LOG_TO_STDOUT as soon as possible.
The old environment variable MINDIE_LLM_LOG_TO_STDOUT will be deprecated on 2025/12/31. Please use the new environment variable MINDIE_LOG_TO_STDOUT as soon as possible.
The old environment variable MINDIE_LLM_PYTHON_LOG_PATH will be deprecated on 2025/12/31. Please use the new environment variable MINDIE_LOG_PATH as soon as possible.
The old environment variable ATB_LOG_TO_FILE will be deprecated on 2025/12/31. Please use the new environment variable MINDIE_LOG_TO_FILE as soon as possible.
The old environment variable MINDIE_LLM_PYTHON_LOG_LEVEL will be deprecated on 2025/12/31. Please use the new environment variable MINDIE_LOG_LEVEL as soon as possible.
The old environment variable MINDIE_LLM_PYTHON_LOG_TO_FILE will be deprecated on 2025/12/31. Please use the new environment variable MINDIE_LOG_TO_FILE as soon as possible.
The old environment variable MINDIE_LLM_LOG_TO_FILE will be deprecated on 2025/12/31. Please use the new environment variable MINDIE_LOG_TO_FILE as soon as possible.
The old environment variable OCK_LOG_LEVEL will be deprecated on 2025/12/31. Please use the new environment variable MINDIE_LOG_LEVEL as soon as possible.
The old environment variable OCK_LOG_TO_STDOUT will be deprecated on 2025/12/31. Please use the new environment variable MINDIE_LOG_TO_STDOUT as soon as possible.
Process ForkServerProcess-5:
Process ForkServerProcess-9:
Process ForkServerProcess-3:
Process ForkServerProcess-4:
Process ForkServerProcess-7:
Process ForkServerProcess-2:
Process ForkServerProcess-8:
Process ForkServerProcess-6:
Traceback (most recent call last):
Traceback (most recent call last):
Traceback (most recent call last):
Traceback (most recent call last):
Traceback (most recent call last):
Traceback (most recent call last):
Traceback (most recent call last):
Traceback (most recent call last):
File "/usr/lib64/python3.11/multiprocessing/managers.py", line 814, in _callmethod
conn = self._tls.connection
^^^^^^^^^^^^^^^^^^^^
File "/usr/lib64/python3.11/multiprocessing/managers.py", line 814, in _callmethod
conn = self._tls.connection
^^^^^^^^^^^^^^^^^^^^
AttributeError: 'ForkAwareLocal' object has no attribute 'connection'
AttributeError: 'ForkAwareLocal' object has no attribute 'connection'
File "/usr/lib64/python3.11/multiprocessing/managers.py", line 814, in _callmethod
conn = self._tls.connection
^^^^^^^^^^^^^^^^^^^^
File "/usr/lib64/python3.11/multiprocessing/managers.py", line 814, in _callmethod
conn = self._tls.connection
^^^^^^^^^^^^^^^^^^^^
File "/usr/lib64/python3.11/multiprocessing/managers.py", line 814, in _callmethod
conn = self._tls.connection
^^^^^^^^^^^^^^^^^^^^
File "/usr/lib64/python3.11/multiprocessing/managers.py", line 814, in _callmethod
conn = self._tls.connection
^^^^^^^^^^^^^^^^^^^^
AttributeError: 'ForkAwareLocal' object has no attribute 'connection'
AttributeError: 'ForkAwareLocal' object has no attribute 'connection'
AttributeError: 'ForkAwareLocal' object has no attribute 'connection'
File "/usr/lib64/python3.11/multiprocessing/managers.py", line 814, in _callmethod
conn = self._tls.connection
^^^^^^^^^^^^^^^^^^^^
During handling of the above exception, another exception occurred:
AttributeError: 'ForkAwareLocal' object has no attribute 'connection'
File "/usr/lib64/python3.11/multiprocessing/managers.py", line 814, in _callmethod
conn = self._tls.connection
^^^^^^^^^^^^^^^^^^^^
Traceback (most recent call last):
AttributeError: 'ForkAwareLocal' object has no attribute 'connection'
During handling of the above exception, another exception occurred:
AttributeError: 'ForkAwareLocal' object has no attribute 'connection'
During handling of the above exception, another exception occurred:
During handling of the above exception, another exception occurred:
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
Traceback (most recent call last):
Traceback (most recent call last):
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
Traceback (most recent call last):
File "/usr/lib64/python3.11/multiprocessing/process.py", line 314, in _bootstrap
self.run()
File "/usr/lib64/python3.11/multiprocessing/process.py", line 108, in run
self._target(*self._args, **self._kwargs)
File "/usr/local/Ascend/ascend-toolkit/latest/python/site-packages/tbe/common/repository_manager/route.py", line 71, in wrapper
raise exp
File "/usr/lib64/python3.11/multiprocessing/process.py", line 314, in _bootstrap
self.run()
File "/usr/lib64/python3.11/multiprocessing/process.py", line 314, in _bootstrap
self.run()
File "/usr/lib64/python3.11/multiprocessing/process.py", line 314, in _bootstrap
self.run()
File "/usr/lib64/python3.11/multiprocessing/process.py", line 314, in _bootstrap
self.run()
File "/usr/local/Ascend/ascend-toolkit/latest/python/site-packages/tbe/common/repository_manager/route.py", line 63, in wrapper
func(*args, **kwargs)
File "/usr/lib64/python3.11/multiprocessing/process.py", line 108, in run
self._target(*self._args, **self._kwargs)
File "/usr/lib64/python3.11/multiprocessing/process.py", line 108, in run
self._target(*self._args, **self._kwargs)
File "/usr/lib64/python3.11/multiprocessing/process.py", line 108, in run
self._target(*self._args, **self._kwargs)
File "/usr/lib64/python3.11/multiprocessing/process.py", line 314, in _bootstrap
self.run()
File "/usr/lib64/python3.11/multiprocessing/process.py", line 314, in _bootstrap
self.run()
File "/usr/local/Ascend/ascend-toolkit/latest/python/site-packages/tbe/common/repository_manager/route.py", line 264, in task_distribute
resource_proxy[SUB_PROCESS_STATE].append(True)
File "/usr/lib64/python3.11/multiprocessing/process.py", line 108, in run
self._target(*self._args, **self._kwargs)
File "/usr/local/Ascend/ascend-toolkit/latest/python/site-packages/tbe/common/repository_manager/route.py", line 71, in wrapper
raise exp
File "/usr/lib64/python3.11/multiprocessing/process.py", line 314, in _bootstrap
self.run()
File "/usr/local/Ascend/ascend-toolkit/latest/python/site-packages/tbe/common/repository_manager/route.py", line 71, in wrapper
raise exp
File "/usr/local/Ascend/ascend-toolkit/latest/python/site-packages/tbe/common/repository_manager/route.py", line 71, in wrapper
raise exp
File "/usr/lib64/python3.11/multiprocessing/process.py", line 108, in run
self._target(*self._args, **self._kwargs)
File "<string>", line 2, in append
File "/usr/lib64/python3.11/multiprocessing/process.py", line 108, in run
self._target(*self._args, **self._kwargs)
File "/usr/local/Ascend/ascend-toolkit/latest/python/site-packages/tbe/common/repository_manager/route.py", line 71, in wrapper
raise exp
File "/usr/local/Ascend/ascend-toolkit/latest/python/site-packages/tbe/common/repository_manager/route.py", line 63, in wrapper
func(*args, **kwargs)
File "/usr/local/Ascend/ascend-toolkit/latest/python/site-packages/tbe/common/repository_manager/route.py", line 63, in wrapper
func(*args, **kwargs)
File "/usr/local/Ascend/ascend-toolkit/latest/python/site-packages/tbe/common/repository_manager/route.py", line 63, in wrapper
func(*args, **kwargs)
File "/usr/lib64/python3.11/multiprocessing/process.py", line 108, in run
self._target(*self._args, **self._kwargs)
File "/usr/local/Ascend/ascend-toolkit/latest/python/site-packages/tbe/common/repository_manager/route.py", line 71, in wrapper
raise exp
File "/usr/lib64/python3.11/multiprocessing/managers.py", line 818, in _callmethod
self._connect()
File "/usr/local/Ascend/ascend-toolkit/latest/python/site-packages/tbe/common/repository_manager/route.py", line 71, in wrapper
raise exp
File "/usr/local/Ascend/ascend-toolkit/latest/python/site-packages/tbe/common/repository_manager/route.py", line 63, in wrapper
func(*args, **kwargs)
File "/usr/local/Ascend/ascend-toolkit/latest/python/site-packages/tbe/common/repository_manager/route.py", line 264, in task_distribute
resource_proxy[SUB_PROCESS_STATE].append(True)
File "/usr/local/Ascend/ascend-toolkit/latest/python/site-packages/tbe/common/repository_manager/route.py", line 264, in task_distribute
resource_proxy[SUB_PROCESS_STATE].append(True)
File "/usr/local/Ascend/ascend-toolkit/latest/python/site-packages/tbe/common/repository_manager/route.py", line 264, in task_distribute
resource_proxy[SUB_PROCESS_STATE].append(True)
File "/usr/local/Ascend/ascend-toolkit/latest/python/site-packages/tbe/common/repository_manager/route.py", line 71, in wrapper
raise exp
File "/usr/local/Ascend/ascend-toolkit/latest/python/site-packages/tbe/common/repository_manager/route.py", line 63, in wrapper
func(*args, **kwargs)
File "/usr/lib64/python3.11/multiprocessing/managers.py", line 805, in _connect
conn = self._Client(self._token.address, authkey=self._authkey)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/Ascend/ascend-toolkit/latest/python/site-packages/tbe/common/repository_manager/route.py", line 63, in wrapper
func(*args, **kwargs)
File "/usr/local/Ascend/ascend-toolkit/latest/python/site-packages/tbe/common/repository_manager/route.py", line 264, in task_distribute
resource_proxy[SUB_PROCESS_STATE].append(True)
File "<string>", line 2, in append
File "<string>", line 2, in append
File "<string>", line 2, in append
File "/usr/local/Ascend/ascend-toolkit/latest/python/site-packages/tbe/common/repository_manager/route.py", line 63, in wrapper
func(*args, **kwargs)
File "/usr/local/Ascend/ascend-toolkit/latest/python/site-packages/tbe/common/repository_manager/route.py", line 264, in task_distribute
resource_proxy[SUB_PROCESS_STATE].append(True)
File "/usr/lib64/python3.11/multiprocessing/connection.py", line 518, in Client
c = SocketClient(address)
^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/Ascend/ascend-toolkit/latest/python/site-packages/tbe/common/repository_manager/route.py", line 264, in task_distribute
resource_proxy[SUB_PROCESS_STATE].append(True)
File "/usr/lib64/python3.11/multiprocessing/managers.py", line 818, in _callmethod
self._connect()
File "/usr/lib64/python3.11/multiprocessing/managers.py", line 818, in _callmethod
self._connect()
File "/usr/lib64/python3.11/multiprocessing/managers.py", line 818, in _callmethod
self._connect()
File "<string>", line 2, in append
File "/usr/local/Ascend/ascend-toolkit/latest/python/site-packages/tbe/common/repository_manager/route.py", line 264, in task_distribute
resource_proxy[SUB_PROCESS_STATE].append(True)
File "<string>", line 2, in append
File "/usr/lib64/python3.11/multiprocessing/connection.py", line 646, in SocketClient
s.connect(address)
File "<string>", line 2, in append
File "/usr/lib64/python3.11/multiprocessing/managers.py", line 805, in _connect
conn = self._Client(self._token.address, authkey=self._authkey)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/lib64/python3.11/multiprocessing/managers.py", line 805, in _connect
conn = self._Client(self._token.address, authkey=self._authkey)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/lib64/python3.11/multiprocessing/managers.py", line 805, in _connect
conn = self._Client(self._token.address, authkey=self._authkey)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/lib64/python3.11/multiprocessing/connection.py", line 518, in Client
c = SocketClient(address)
^^^^^^^^^^^^^^^^^^^^^
File "/usr/lib64/python3.11/multiprocessing/managers.py", line 818, in _callmethod
self._connect()
File "<string>", line 2, in append
File "/usr/lib64/python3.11/multiprocessing/managers.py", line 818, in _callmethod
self._connect()
File "/usr/lib64/python3.11/multiprocessing/managers.py", line 818, in _callmethod
self._connect()
ConnectionRefusedError: [Errno 111] Connection refused
File "/usr/lib64/python3.11/multiprocessing/connection.py", line 518, in Client
c = SocketClient(address)
^^^^^^^^^^^^^^^^^^^^^
File "/usr/lib64/python3.11/multiprocessing/connection.py", line 518, in Client
c = SocketClient(address)
^^^^^^^^^^^^^^^^^^^^^
File "/usr/lib64/python3.11/multiprocessing/connection.py", line 646, in SocketClient
s.connect(address)
File "/usr/lib64/python3.11/multiprocessing/managers.py", line 805, in _connect
conn = self._Client(self._token.address, authkey=self._authkey)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/lib64/python3.11/multiprocessing/managers.py", line 818, in _callmethod
self._connect()
File "/usr/lib64/python3.11/multiprocessing/managers.py", line 805, in _connect
conn = self._Client(self._token.address, authkey=self._authkey)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/lib64/python3.11/multiprocessing/managers.py", line 805, in _connect
conn = self._Client(self._token.address, authkey=self._authkey)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/lib64/python3.11/multiprocessing/connection.py", line 646, in SocketClient
s.connect(address)
File "/usr/lib64/python3.11/multiprocessing/connection.py", line 646, in SocketClient
s.connect(address)
ConnectionRefusedError: [Errno 111] Connection refused
File "/usr/lib64/python3.11/multiprocessing/connection.py", line 518, in Client
c = SocketClient(address)
^^^^^^^^^^^^^^^^^^^^^
File "/usr/lib64/python3.11/multiprocessing/managers.py", line 805, in _connect
conn = self._Client(self._token.address, authkey=self._authkey)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/lib64/python3.11/multiprocessing/connection.py", line 518, in Client
c = SocketClient(address)
^^^^^^^^^^^^^^^^^^^^^
File "/usr/lib64/python3.11/multiprocessing/connection.py", line 518, in Client
c = SocketClient(address)
^^^^^^^^^^^^^^^^^^^^^
ConnectionRefusedError: [Errno 111] Connection refused
ConnectionRefusedError: [Errno 111] Connection refused
File "/usr/lib64/python3.11/multiprocessing/connection.py", line 646, in SocketClient
s.connect(address)
File "/usr/lib64/python3.11/multiprocessing/connection.py", line 518, in Client
c = SocketClient(address)
^^^^^^^^^^^^^^^^^^^^^
File "/usr/lib64/python3.11/multiprocessing/connection.py", line 646, in SocketClient
s.connect(address)
File "/usr/lib64/python3.11/multiprocessing/connection.py", line 646, in SocketClient
s.connect(address)
File "/usr/lib64/python3.11/multiprocessing/connection.py", line 646, in SocketClient
s.connect(address)
ConnectionRefusedError: [Errno 111] Connection refused
ConnectionRefusedError: [Errno 111] Connection refused
ConnectionRefusedError: [Errno 111] Connection refused
ConnectionRefusedError: [Errno 111] Connection refused
/usr/lib64/python3.11/multiprocessing/resource_tracker.py:254: UserWarning: resource_tracker: There appear to be 30 leaked semaphore objects to clean up at shutdown
warnings.warn('resource_tracker: There appear to be %d '
Daemon is killing...
Killed