lsn@lsn:~/infer_project$ atc --model=yolov5s.onnx --framework=5 --output=model --soc_version=Ascend310B4
ATC start working now, please wait for a moment.
Exception in thread Thread-1:
Traceback (most recent call last):
File "/usr/lib/python3.8/threading.py", line 932, in _bootstrap_inner
self.run()
File "/home/lsn/Ascend/ascend-toolkit/latest/python/site-packages/tbe/common/repository_manager/utils/multiprocess_util.py", line 91, in run
key, func, args, kwargs = self.task_q.get(timeout=TIMEOUT)
File "<string>", line 2, in get
File "/usr/lib/python3.8/multiprocessing/managers.py", line 835, in _callmethod
kind, result = conn.recv()
File "/usr/lib/python3.8/multiprocessing/connection.py", line 250, in recv
buf = self._recv_bytes()
File "/usr/lib/python3.8/multiprocessing/connection.py", line 414, in _recv_bytes
buf = self._recv(4)
File "/usr/lib/python3.8/multiprocessing/connection.py", line 383, in _recv
raise EOFError
EOFError
Traceback (most recent call last):
File "/usr/lib/python3.8/multiprocessing/util.py", line 300, in _run_finalizers
finalizer()
File "/usr/lib/python3.8/multiprocessing/util.py", line 224, in __call__
res = self._callback(*self._args, **self._kwargs)
File "/home/lsn/Ascend/ascend-toolkit/latest/python/site-packages/tbe/common/repository_manager/route.py", line 51, in wrapper
return func(cls, *args, **kwargs)
File "/home/lsn/Ascend/ascend-toolkit/latest/python/site-packages/tbe/common/repository_manager/route.py", line 207, in finalize
cls.bank_process_pool.close()
AttributeError: 'NoneType' object has no attribute 'close'
...
ATC run failed, Please check the detail log, Try 'atc --help' for more information
E40021: Failed to compile Op [Conv_272,[Conv_272,Conv_272]]. (oppath: [Compile /home/lsn/Ascend/ascend-toolkit/7.0.1/opp/built-in/op_impl/ai_core/tbe/impl/conv2d.py failed with errormsg/stack: File "/usr/lib/python3.8/multiprocessing/connection.py", line 630, in SocketClient
s.connect(address)
ConnectionRefusedError: [Errno 111] Connection refused
], optype: [Conv2D])
Solution: See the host log for details, and then check the Python stack where the error log is reported.
TraceBack (most recent call last):
Compile op[Conv_272,[Conv_272,Conv_272]] failed, oppath[/home/lsn/Ascend/ascend-toolkit/7.0.1/opp/built-in/op_impl/ai_core/tbe/impl/conv2d.py], optype[Conv2D], taskID[270]. Please check op's compilation error message.[FUNC:ReportBuildErrMessage][FILE:fusion_manager.cc][LINE:771]
Failed to compile Op [Conv_235,[Conv_235,Conv_235]]. (oppath: [Compile /home/lsn/Ascend/ascend-toolkit/7.0.1/opp/built-in/op_impl/ai_core/tbe/impl/conv2d.py failed with errormsg/stack: File "/usr/lib/python3.8/multiprocessing/connection.py", line 630, in SocketClient
s.connect(address)
ConnectionRefusedError: [Errno 111] Connection refused
], optype: [Conv2D])
Compile op[Conv_235,[Conv_235,Conv_235]] failed, oppath[/home/lsn/Ascend/ascend-toolkit/7.0.1/opp/built-in/op_impl/ai_core/tbe/impl/conv2d.py], optype[Conv2D], taskID[274]. Please check op's compilation error message.[FUNC:ReportBuildErrMessage][FILE:fusion_manager.cc][LINE:771]
Failed to compile Op [Conv_198,[Conv_198,Conv_198]]. (oppath: [Compile /home/lsn/Ascend/ascend-toolkit/7.0.1/opp/built-in/op_impl/ai_core/tbe/impl/conv2d.py failed with errormsg/stack: File "/usr/lib/python3.8/multiprocessing/connection.py", line 630, in SocketClient
s.connect(address)
ConnectionRefusedError: [Errno 111] Connection refused
], optype: [Conv2D])
Compile op[Conv_198,[Conv_198,Conv_198]] failed, oppath[/home/lsn/Ascend/ascend-toolkit/7.0.1/opp/built-in/op_impl/ai_core/tbe/impl/conv2d.py], optype[Conv2D], taskID[278]. Please check op's compilation error message.[FUNC:ReportBuildErrMessage][FILE:fusion_manager.cc][LINE:771]
Failed to compile Op [Conv_133,[Conv_133,Conv_133]]. (oppath: [Compile /home/lsn/Ascend/ascend-toolkit/7.0.1/opp/built-in/op_impl/ai_core/tbe/impl/conv2d.py failed with errormsg/stack: File "/usr/lib/python3.8/multiprocessing/connection.py", line 630, in SocketClient
s.connect(address)
ConnectionRefusedError: [Errno 111] Connection refused
], optype: [Conv2D])
Compile op[Conv_133,[Conv_133,Conv_133]] failed, oppath[/home/lsn/Ascend/ascend-toolkit/7.0.1/opp/built-in/op_impl/ai_core/tbe/impl/conv2d.py], optype[Conv2D], taskID[350]. Please check op's compilation error message.[FUNC:ReportBuildErrMessage][FILE:fusion_manager.cc][LINE:771]
Failed to compile Op [Conv_123,[Conv_123,Conv_123]]. (oppath: [Compile /home/lsn/Ascend/ascend-toolkit/7.0.1/opp/built-in/op_impl/ai_core/tbe/impl/conv2d.py failed with errormsg/stack: File "/usr/lib/python3.8/multiprocessing/connection.py", line 630, in SocketClient
s.connect(address)
ConnectionRefusedError: [Errno 111] Connection refused
], optype: [Conv2D])
Compile op[Conv_123,[Conv_123,Conv_123]] failed, oppath[/home/lsn/Ascend/ascend-toolkit/7.0.1/opp/built-in/op_impl/ai_core/tbe/impl/conv2d.py], optype[Conv2D], taskID[353]. Please check op's compilation error message.[FUNC:ReportBuildErrMessage][FILE:fusion_manager.cc][LINE:771]
Failed to compile Op [Conv_126,[Conv_126,Conv_126]]. (oppath: [Compile /home/lsn/Ascend/ascend-toolkit/7.0.1/opp/built-in/op_impl/ai_core/tbe/impl/conv2d.py failed with errormsg/stack: File "/usr/lib/python3.8/multiprocessing/connection.py", line 630, in SocketClient
s.connect(address)
ConnectionRefusedError: [Errno 111] Connection refused
], optype: [Conv2D])
Compile op[Conv_126,[Conv_126,Conv_126]] failed, oppath[/home/lsn/Ascend/ascend-toolkit/7.0.1/opp/built-in/op_impl/ai_core/tbe/impl/conv2d.py], optype[Conv2D], taskID[356]. Please check op's compilation error message.[FUNC:ReportBuildErrMessage][FILE:fusion_manager.cc][LINE:771]
Failed to compile Op [Conv_136,[Conv_136,Conv_136]]. (oppath: [Compile /home/lsn/Ascend/ascend-toolkit/7.0.1/opp/built-in/op_impl/ai_core/tbe/impl/conv2d.py failed with errormsg/stack: File "/usr/lib/python3.8/multiprocessing/connection.py", line 630, in SocketClient
s.connect(address)
ConnectionRefusedError: [Errno 111] Connection refused
], optype: [Conv2D])
Compile op[Conv_136,[Conv_136,Conv_136]] failed, oppath[/home/lsn/Ascend/ascend-toolkit/7.0.1/opp/built-in/op_impl/ai_core/tbe/impl/conv2d.py], optype[Conv2D], taskID[365]. Please check op's compilation error message.[FUNC:ReportBuildErrMessage][FILE:fusion_manager.cc][LINE:771]
Failed to compile Op [Conv_120,[Conv_120,Conv_120]]. (oppath: [Compile /home/lsn/Ascend/ascend-toolkit/7.0.1/opp/built-in/op_impl/ai_core/tbe/impl/conv2d.py failed with errormsg/stack: File "/usr/lib/python3.8/multiprocessing/connection.py", line 630, in SocketClient
s.connect(address)
ConnectionRefusedError: [Errno 111] Connection refused
], optype: [Conv2D])
Compile op[Conv_120,[Conv_120,Conv_120]] failed, oppath[/home/lsn/Ascend/ascend-toolkit/7.0.1/opp/built-in/op_impl/ai_core/tbe/impl/conv2d.py], optype[Conv2D], taskID[359]. Please check op's compilation error message.[FUNC:ReportBuildErrMessage][FILE:fusion_manager.cc][LINE:771]
Failed to compile Op [Conv_84,[Conv_84,Conv_84]]. (oppath: [Compile /home/lsn/Ascend/ascend-toolkit/7.0.1/opp/built-in/op_impl/ai_core/tbe/impl/conv2d.py failed with errormsg/stack: File "/usr/lib/python3.8/multiprocessing/connection.py", line 630, in SocketClient
s.connect(address)
ConnectionRefusedError: [Errno 111] Connection refused
], optype: [Conv2D])
Compile op[Conv_84,[Conv_84,Conv_84]] failed, oppath[/home/lsn/Ascend/ascend-toolkit/7.0.1/opp/built-in/op_impl/ai_core/tbe/impl/conv2d.py], optype[Conv2D], taskID[368]. Please check op's compilation error message.[FUNC:ReportBuildErrMessage][FILE:fusion_manager.cc][LINE:771]
Failed to compile Op [Conv_26,[Conv_26,Conv_26]]. (oppath: [Compile /home/lsn/Ascend/ascend-toolkit/7.0.1/opp/built-in/op_impl/ai_core/tbe/impl/conv2d.py failed with errormsg/stack: File "/usr/lib/python3.8/multiprocessing/connection.py", line 630, in SocketClient
s.connect(address)
ConnectionRefusedError: [Errno 111] Connection refused
], optype: [Conv2D])
Compile op[Conv_26,[Conv_26,Conv_26]] failed, oppath[/home/lsn/Ascend/ascend-toolkit/7.0.1/opp/built-in/op_impl/ai_core/tbe/impl/conv2d.py], optype[Conv2D], taskID[371]. Please check op's compilation error message.[FUNC:ReportBuildErrMessage][FILE:fusion_manager.cc][LINE:771]
Failed to compile Op [Conv_111,[Conv_111,Conv_111]]. (oppath: [Compile /home/lsn/Ascend/ascend-toolkit/7.0.1/opp/built-in/op_impl/ai_core/tbe/impl/conv2d.py failed with errormsg/stack: File "/usr/lib/python3.8/multiprocessing/connection.py", line 630, in SocketClient
s.connect(address)
ConnectionRefusedError: [Errno 111] Connection refused
], optype: [Conv2D])
Compile op[Conv_111,[Conv_111,Conv_111]] failed, oppath[/home/lsn/Ascend/ascend-toolkit/7.0.1/opp/built-in/op_impl/ai_core/tbe/impl/conv2d.py], optype[Conv2D], taskID[377]. Please check op's compilation error message.[FUNC:ReportBuildErrMessage][FILE:fusion_manager.cc][LINE:771]
Failed to compile Op [Conv_50,[Conv_50,Conv_50]]. (oppath: [Compile /home/lsn/Ascend/ascend-toolkit/7.0.1/opp/built-in/op_impl/ai_core/tbe/impl/conv2d.py failed with errormsg/stack: File "/usr/lib/python3.8/multiprocessing/connection.py", line 630, in SocketClient
s.connect(address)
ConnectionRefusedError: [Errno 111] Connection refused
], optype: [Conv2D])
Compile op[Conv_50,[Conv_50,Conv_50]] failed, oppath[/home/lsn/Ascend/ascend-toolkit/7.0.1/opp/built-in/op_impl/ai_core/tbe/impl/conv2d.py], optype[Conv2D], taskID[380]. Please check op's compilation error message.[FUNC:ReportBuildErrMessage][FILE:fusion_manager.cc][LINE:771]
Failed to compile Op [Conv_36,[Conv_36,Conv_36]]. (oppath: [Compile /home/lsn/Ascend/ascend-toolkit/7.0.1/opp/built-in/op_impl/ai_core/tbe/impl/conv2d.py failed with errormsg/stack: File "/usr/lib/python3.8/multiprocessing/connection.py", line 630, in SocketClient
s.connect(address)
ConnectionRefusedError: [Errno 111] Connection refused
], optype: [Conv2D])
Compile op[Conv_36,[Conv_36,Conv_36]] failed, oppath[/home/lsn/Ascend/ascend-toolkit/7.0.1/opp/built-in/op_impl/ai_core/tbe/impl/conv2d.py], optype[Conv2D], taskID[383]. Please check op's compilation error message.[FUNC:ReportBuildErrMessage][FILE:fusion_manager.cc][LINE:771]
Failed to compile Op [Conv_39,[Conv_39,Conv_39]]. (oppath: [Compile /home/lsn/Ascend/ascend-toolkit/7.0.1/opp/built-in/op_impl/ai_core/tbe/impl/conv2d.py failed with errormsg/stack: File "/usr/lib/python3.8/multiprocessing/connection.py", line 630, in SocketClient
s.connect(address)
ConnectionRefusedError: [Errno 111] Connection refused
], optype: [Conv2D])
Compile op[Conv_39,[Conv_39,Conv_39]] failed, oppath[/home/lsn/Ascend/ascend-toolkit/7.0.1/opp/built-in/op_impl/ai_core/tbe/impl/conv2d.py], optype[Conv2D], taskID[386]. Please check op's compilation error message.[FUNC:ReportBuildErrMessage][FILE:fusion_manager.cc][LINE:771]
[SubGraphOpt][Compile][ProcFailedCompTask] Thread[140026226648832] recompile single op[Conv_129] failed[FUNC:ProcessAllFailedCompileTasks][FILE:tbe_op_store_adapter.cc][LINE:954]
[SubGraphOpt][Compile][ProcFailedCompTask] Thread[140026226648832] recompile single op[Conv_120] failed[FUNC:ProcessAllFailedCompileTasks][FILE:tbe_op_store_adapter.cc][LINE:954]
[SubGraphOpt][Compile][ProcFailedCompTask] Thread[140026226648832] recompile single op[Conv_136] failed[FUNC:ProcessAllFailedCompileTasks][FILE:tbe_op_store_adapter.cc][LINE:954]
[SubGraphOpt][Compile][ProcFailedCompTask] Thread[140026226648832] recompile single op[Conv_126] failed[FUNC:ProcessAllFailedCompileTasks][FILE:tbe_op_store_adapter.cc][LINE:954]
[SubGraphOpt][Compile][ProcFailedCompTask] Thread[140026226648832] recompile single op[Conv_123] failed[FUNC:ProcessAllFailedCompileTasks][FILE:tbe_op_store_adapter.cc][LINE:954]
[SubGraphOpt][Compile][ProcFailedCompTask] Thread[140026226648832] recompile single op[Conv_133] failed[FUNC:ProcessAllFailedCompileTasks][FILE:tbe_op_store_adapter.cc][LINE:954]
[SubGraphOpt][Compile][ParalCompOp] Thread[140026226648832] process fail task failed[FUNC:ParallelCompileOp][FILE:tbe_op_store_adapter.cc][LINE:1001]
[SubGraphOpt][Compile][CompOpOnly] CompileOp failed.[FUNC:CompileOpOnly][FILE:op_compiler.cc][LINE:1127]
[GraphOpt][FusedGraph][RunCompile] Failed to compile graph with compiler Normal mode Op Compiler[FUNC:SubGraphCompile][FILE:fe_graph_optimizer.cc][LINE:1292]
Call OptimizeFusedGraph failed, ret:-1, engine_name:AIcoreEngine, graph_name:partition0_rank3_new_sub_graph133[FUNC:OptimizeSubGraph][FILE:graph_optimize.cc][LINE:131]
Failed to compile Op [Conv_47,[Conv_47,Conv_47]]. (oppath: [Compile /home/lsn/Ascend/ascend-toolkit/7.0.1/opp/built-in/op_impl/ai_core/tbe/impl/conv2d.py failed with errormsg/stack: File "/usr/lib/python3.8/multiprocessing/connection.py", line 630, in SocketClient
s.connect(address)
ConnectionRefusedError: [Errno 111] Connection refused
], optype: [Conv2D])
Compile op[Conv_47,[Conv_47,Conv_47]] failed, oppath[/home/lsn/Ascend/ascend-toolkit/7.0.1/opp/built-in/op_impl/ai_core/tbe/impl/conv2d.py], optype[Conv2D], taskID[422]. Please check op's compilation error message.[FUNC:ReportBuildErrMessage][FILE:fusion_manager.cc][LINE:771]
Failed to compile Op [Conv_188,[Conv_188,Conv_188]]. (oppath: [Compile /home/lsn/Ascend/ascend-toolkit/7.0.1/opp/built-in/op_impl/ai_core/tbe/impl/conv2d.py failed with errormsg/stack: File "/usr/lib/python3.8/multiprocessing/connection.py", line 630, in SocketClient
s.connect(address)
ConnectionRefusedError: [Errno 111] Connection refused
], optype: [Conv2D])
Compile op[Conv_188,[Conv_188,Conv_188]] failed, oppath[/home/lsn/Ascend/ascend-toolkit/7.0.1/opp/built-in/op_impl/ai_core/tbe/impl/conv2d.py], optype[Conv2D], taskID[428]. Please check op's compilation error message.[FUNC:ReportBuildErrMessage][FILE:fusion_manager.cc][LINE:771]
Failed to compile Op [Conv_185,[Conv_185,Conv_185]]. (oppath: [Compile /home/lsn/Ascend/ascend-toolkit/7.0.1/opp/built-in/op_impl/ai_core/tbe/impl/conv2d.py failed with errormsg/stack: File "/usr/lib/python3.8/multiprocessing/connection.py", line 630, in SocketClient
s.connect(address)
ConnectionRefusedError: [Errno 111] Connection refused
], optype: [Conv2D])
Compile op[Conv_185,[Conv_185,Conv_185]] failed, oppath[/home/lsn/Ascend/ascend-toolkit/7.0.1/opp/built-in/op_impl/ai_core/tbe/impl/conv2d.py], optype[Conv2D], taskID[431]. Please check op's compilation error message.[FUNC:ReportBuildErrMessage][FILE:fusion_manager.cc][LINE:771]
Failed to compile Op [Conv_195,[Conv_195,Conv_195]]. (oppath: [Compile /home/lsn/Ascend/ascend-toolkit/7.0.1/opp/built-in/op_impl/ai_core/tbe/impl/conv2d.py failed with errormsg/stack: File "/usr/lib/python3.8/multiprocessing/connection.py", line 630, in SocketClient
s.connect(address)
ConnectionRefusedError: [Errno 111] Connection refused
], optype: [Conv2D])
Compile op[Conv_195,[Conv_195,Conv_195]] failed, oppath[/home/lsn/Ascend/ascend-toolkit/7.0.1/opp/built-in/op_impl/ai_core/tbe/impl/conv2d.py], optype[Conv2D], taskID[425]. Please check op's compilation error message.[FUNC:ReportBuildErrMessage][FILE:fusion_manager.cc][LINE:771]
Failed to compile Op [Conv_178,[Conv_178,Conv_178]]. (oppath: [Compile /home/lsn/Ascend/ascend-toolkit/7.0.1/opp/built-in/op_impl/ai_core/tbe/impl/conv2d.py failed with errormsg/stack: File "/usr/lib/python3.8/multiprocessing/connection.py", line 630, in SocketClient
s.connect(address)
ConnectionRefusedError: [Errno 111] Connection refused
], optype: [Conv2D])
Compile op[Conv_178,[Conv_178,Conv_178]] failed, oppath[/home/lsn/Ascend/ascend-toolkit/7.0.1/opp/built-in/op_impl/ai_core/tbe/impl/conv2d.py], optype[Conv2D], taskID[434]. Please check op's compilation error message.[FUNC:ReportBuildErrMessage][FILE:fusion_manager.cc][LINE:771]
Failed to compile Op [Conv_182,[Conv_182,Conv_182]]. (oppath: [Compile /home/lsn/Ascend/ascend-toolkit/7.0.1/opp/built-in/op_impl/ai_core/tbe/impl/conv2d.py failed with errormsg/stack: File "/usr/lib/python3.8/multiprocessing/connection.py", line 630, in SocketClient
s.connect(address)
ConnectionRefusedError: [Errno 111] Connection refused
], optype: [Conv2D])
Compile op[Conv_182,[Conv_182,Conv_182]] failed, oppath[/home/lsn/Ascend/ascend-toolkit/7.0.1/opp/built-in/op_impl/ai_core/tbe/impl/conv2d.py], optype[Conv2D], taskID[437]. Please check op's compilation error message.[FUNC:ReportBuildErrMessage][FILE:fusion_manager.cc][LINE:771]
Failed to compile Op [Conv_16,[Conv_16,Conv_16]]. (oppath: [Compile /home/lsn/Ascend/ascend-toolkit/7.0.1/opp/built-in/op_impl/ai_core/tbe/impl/conv2d.py failed with errormsg/stack: File "/usr/lib/python3.8/multiprocessing/connection.py", line 630, in SocketClient
s.connect(address)
ConnectionRefusedError: [Errno 111] Connection refused
], optype: [Conv2D])
Compile op[Conv_16,[Conv_16,Conv_16]] failed, oppath[/home/lsn/Ascend/ascend-toolkit/7.0.1/opp/built-in/op_impl/ai_core/tbe/impl/conv2d.py], optype[Conv2D], taskID[448]. Please check op's compilation error message.[FUNC:ReportBuildErrMessage][FILE:fusion_manager.cc][LINE:771]
Failed to compile Op [Conv_9,[Conv_9,Conv_9]]. (oppath: [Compile /home/lsn/Ascend/ascend-toolkit/7.0.1/opp/built-in/op_impl/ai_core/tbe/impl/conv2d.py failed with errormsg/stack: File "/usr/lib/python3.8/multiprocessing/connection.py", line 630, in SocketClient
s.connect(address)
ConnectionRefusedError: [Errno 111] Connection refused
], optype: [Conv2D])
Compile op[Conv_9,[Conv_9,Conv_9]] failed, oppath[/home/lsn/Ascend/ascend-toolkit/7.0.1/opp/built-in/op_impl/ai_core/tbe/impl/conv2d.py], optype[Conv2D], taskID[454]. Please check op's compilation error message.[FUNC:ReportBuildErrMessage][FILE:fusion_manager.cc][LINE:771]
Failed to compile Op [Conv_0,[Conv_0,Conv_0]]. (oppath: [Compile /home/lsn/Ascend/ascend-toolkit/7.0.1/opp/built-in/op_impl/ai_core/tbe/impl/conv2d.py failed with errormsg/stack: File "/usr/lib/python3.8/multiprocessing/connection.py", line 630, in SocketClient
s.connect(address)
ConnectionRefusedError: [Errno 111] Connection refused
], optype: [Conv2D])
Compile op[Conv_0,[Conv_0,Conv_0]] failed, oppath[/home/lsn/Ascend/ascend-toolkit/7.0.1/opp/built-in/op_impl/ai_core/tbe/impl/conv2d.py], optype[Conv2D], taskID[464]. Please check op's compilation error message.[FUNC:ReportBuildErrMessage][FILE:fusion_manager.cc][LINE:771]
Failed to compile Op [Conv_158,[Conv_158,Conv_158]]. (oppath: [Compile /home/lsn/Ascend/ascend-toolkit/7.0.1/opp/built-in/op_impl/ai_core/tbe/impl/conv2d.py failed with errormsg/stack: File "/usr/lib/python3.8/multiprocessing/connection.py", line 630, in SocketClient
s.connect(address)
ConnectionRefusedError: [Errno 111] Connection refused
], optype: [Conv2D])
Compile op[Conv_158,[Conv_158,Conv_158]] failed, oppath[/home/lsn/Ascend/ascend-toolkit/7.0.1/opp/built-in/op_impl/ai_core/tbe/impl/conv2d.py], optype[Conv2D], taskID[471]. Please check op's compilation error message.[FUNC:ReportBuildErrMessage][FILE:fusion_manager.cc][LINE:771]
Failed to compile Op [Conv_12,[Conv_12,Conv_12]]. (oppath: [Compile /home/lsn/Ascend/ascend-toolkit/7.0.1/opp/built-in/op_impl/ai_core/tbe/impl/conv2d.py failed with errormsg/stack: File "/usr/lib/python3.8/multiprocessing/connection.py", line 630, in SocketClient
s.connect(address)
ConnectionRefusedError: [Errno 111] Connection refused
], optype: [Conv2D])
Compile op[Conv_12,[Conv_12,Conv_12]] failed, oppath[/home/lsn/Ascend/ascend-toolkit/7.0.1/opp/built-in/op_impl/ai_core/tbe/impl/conv2d.py], optype[Conv2D], taskID[496]. Please check op's compilation error message.[FUNC:ReportBuildErrMessage][FILE:fusion_manager.cc][LINE:771]
Failed to compile Op [Conv_3,[Conv_3,Conv_3]]. (oppath: [Compile /home/lsn/Ascend/ascend-toolkit/7.0.1/opp/built-in/op_impl/ai_core/tbe/impl/conv2d.py failed with errormsg/stack: File "/usr/lib/python3.8/multiprocessing/connection.py", line 630, in SocketClient
s.connect(address)
ConnectionRefusedError: [Errno 111] Connection refused
], optype: [Conv2D])
Compile op[Conv_3,[Conv_3,Conv_3]] failed, oppath[/home/lsn/Ascend/ascend-toolkit/7.0.1/opp/built-in/op_impl/ai_core/tbe/impl/conv2d.py], optype[Conv2D], taskID[490]. Please check op's compilation error message.[FUNC:ReportBuildErrMessage][FILE:fusion_manager.cc][LINE:771]
Failed to compile Op [Conv_20,[Conv_20,Conv_20]]. (oppath: [Compile /home/lsn/Ascend/ascend-toolkit/7.0.1/opp/built-in/op_impl/ai_core/tbe/impl/conv2d.py failed with errormsg/stack: File "/usr/lib/python3.8/multiprocessing/connection.py", line 630, in SocketClient
s.connect(address)
ConnectionRefusedError: [Errno 111] Connection refused
], optype: [Conv2D])
Compile op[Conv_20,[Conv_20,Conv_20]] failed, oppath[/home/lsn/Ascend/ascend-toolkit/7.0.1/opp/built-in/op_impl/ai_core/tbe/impl/conv2d.py], optype[Conv2D], taskID[499]. Please check op's compilation error message.[FUNC:ReportBuildErrMessage][FILE:fusion_manager.cc][LINE:771]
Failed to compile Op [Conv_142,[Conv_142,Conv_142]]. (oppath: [Compile /home/lsn/Ascend/ascend-toolkit/7.0.1/opp/built-in/op_impl/ai_core/tbe/impl/conv2d.py failed with errormsg/stack: File "/usr/lib/python3.8/multiprocessing/connection.py", line 630, in SocketClient
s.connect(address)
ConnectionRefusedError: [Errno 111] Connection refused
], optype: [Conv2D])
Compile op[Conv_142,[Conv_142,Conv_142]] failed, oppath[/home/lsn/Ascend/ascend-toolkit/7.0.1/opp/built-in/op_impl/ai_core/tbe/impl/conv2d.py], optype[Conv2D], taskID[504]. Please check op's compilation error message.[FUNC:ReportBuildErrMessage][FILE:fusion_manager.cc][LINE:771]
[SubGraphOpt][Compile][ProcFailedCompTask] Thread[140026251826944] recompile single op[Conv_151] failed[FUNC:ProcessAllFailedCompileTasks][FILE:tbe_op_store_adapter.cc][LINE:954]
[SubGraphOpt][Compile][ProcFailedCompTask] Thread[140026251826944] recompile single op[Conv_142] failed[FUNC:ProcessAllFailedCompileTasks][FILE:tbe_op_store_adapter.cc][LINE:954]
[SubGraphOpt][Compile][ProcFailedCompTask] Thread[140026251826944] recompile single op[Conv_158] failed[FUNC:ProcessAllFailedCompileTasks][FILE:tbe_op_store_adapter.cc][LINE:954]
[SubGraphOpt][Compile][ProcFailedCompTask] Thread[140026251826944] recompile single op[Conv_191] failed[FUNC:ProcessAllFailedCompileTasks][FILE:tbe_op_store_adapter.cc][LINE:954]
[SubGraphOpt][Compile][ProcFailedCompTask] Thread[140026251826944] recompile single op[Conv_182] failed[FUNC:ProcessAllFailedCompileTasks][FILE:tbe_op_store_adapter.cc][LINE:954]
[SubGraphOpt][Compile][ProcFailedCompTask] Thread[140026251826944] recompile single op[Conv_185] failed[FUNC:ProcessAllFailedCompileTasks][FILE:tbe_op_store_adapter.cc][LINE:954]
[SubGraphOpt][Compile][ProcFailedCompTask] Thread[140026251826944] recompile single op[Conv_188] failed[FUNC:ProcessAllFailedCompileTasks][FILE:tbe_op_store_adapter.cc][LINE:954]
[SubGraphOpt][Compile][ProcFailedCompTask] Thread[140026251826944] recompile single op[Conv_178] failed[FUNC:ProcessAllFailedCompileTasks][FILE:tbe_op_store_adapter.cc][LINE:954]
[SubGraphOpt][Compile][ProcFailedCompTask] Thread[140026251826944] recompile single op[Conv_155] failed[FUNC:ProcessAllFailedCompileTasks][FILE:tbe_op_store_adapter.cc][LINE:954]
[SubGraphOpt][Compile][ProcFailedCompTask] Thread[140026251826944] recompile single op[Conv_148] failed[FUNC:ProcessAllFailedCompileTasks][FILE:tbe_op_store_adapter.cc][LINE:954]
[SubGraphOpt][Compile][ProcFailedCompTask] Thread[140026251826944] recompile single op[Conv_145] failed[FUNC:ProcessAllFailedCompileTasks][FILE:tbe_op_store_adapter.cc][LINE:954]
[SubGraphOpt][Compile][ProcFailedCompTask] Thread[140026251826944] recompile single op[Conv_235] failed[FUNC:ProcessAllFailedCompileTasks][FILE:tbe_op_store_adapter.cc][LINE:954]
[SubGraphOpt][Compile][ProcFailedCompTask] Thread[140026251826944] recompile single op[Conv_272] failed[FUNC:ProcessAllFailedCompileTasks][FILE:tbe_op_store_adapter.cc][LINE:954]
[SubGraphOpt][Compile][ProcFailedCompTask] Thread[140026251826944] recompile single op[Conv_175] failed[FUNC:ProcessAllFailedCompileTasks][FILE:tbe_op_store_adapter.cc][LINE:954]
[SubGraphOpt][Compile][ProcFailedCompTask] Thread[140026251826944] recompile single op[Conv_165] failed[FUNC:ProcessAllFailedCompileTasks][FILE:tbe_op_store_adapter.cc][LINE:954]
[SubGraphOpt][Compile][ProcFailedCompTask] Thread[140026251826944] recompile single op[Conv_195] failed[FUNC:ProcessAllFailedCompileTasks][FILE:tbe_op_store_adapter.cc][LINE:954]
[SubGraphOpt][Compile][ProcFailedCompTask] Thread[140026251826944] recompile single op[Conv_198] failed[FUNC:ProcessAllFailedCompileTasks][FILE:tbe_op_store_adapter.cc][LINE:954]
[SubGraphOpt][Compile][ProcFailedCompTask] Thread[140026251826944] recompile single op[Conv_168] failed[FUNC:ProcessAllFailedCompileTasks][FILE:tbe_op_store_adapter.cc][LINE:954]
[SubGraphOpt][Compile][ProcFailedCompTask] Thread[140026251826944] recompile single op[Conv_162] failed[FUNC:ProcessAllFailedCompileTasks][FILE:tbe_op_store_adapter.cc][LINE:954]
[SubGraphOpt][Compile][ProcFailedCompTask] Thread[140026251826944] recompile single op[Conv_171] failed[FUNC:ProcessAllFailedCompileTasks][FILE:tbe_op_store_adapter.cc][LINE:954]
[SubGraphOpt][Compile][ParalCompOp] Thread[140026251826944] process fail task failed[FUNC:ParallelCompileOp][FILE:tbe_op_store_adapter.cc][LINE:1001]
Call OptimizeFusedGraph failed, ret:-1, engine_name:AIcoreEngine, graph_name:partition0_rank5_new_sub_graph135[FUNC:OptimizeSubGraph][FILE:graph_optimize.cc][LINE:131]
Failed to compile Op [Conv_23,[Conv_23,Conv_23]]. (oppath: [Compile /home/lsn/Ascend/ascend-toolkit/7.0.1/opp/built-in/op_impl/ai_core/tbe/impl/conv2d.py failed with errormsg/stack: File "/usr/lib/python3.8/multiprocessing/connection.py", line 630, in SocketClient
s.connect(address)
ConnectionRefusedError: [Errno 111] Connection refused
], optype: [Conv2D])
Compile op[Conv_23,[Conv_23,Conv_23]] failed, oppath[/home/lsn/Ascend/ascend-toolkit/7.0.1/opp/built-in/op_impl/ai_core/tbe/impl/conv2d.py], optype[Conv2D], taskID[516]. Please check op's compilation error message.[FUNC:ReportBuildErrMessage][FILE:fusion_manager.cc][LINE:771]
[SubGraphOpt][Compile][ProcFailedCompTask] Thread[140025572345600] recompile single op[Conv_20] failed[FUNC:ProcessAllFailedCompileTasks][FILE:tbe_op_store_adapter.cc][LINE:954]
[SubGraphOpt][Compile][ProcFailedCompTask] Thread[140025572345600] recompile single op[Conv_3] failed[FUNC:ProcessAllFailedCompileTasks][FILE:tbe_op_store_adapter.cc][LINE:954]
[SubGraphOpt][Compile][ProcFailedCompTask] Thread[140025572345600] recompile single op[Conv_12] failed[FUNC:ProcessAllFailedCompileTasks][FILE:tbe_op_store_adapter.cc][LINE:954]
[SubGraphOpt][Compile][ProcFailedCompTask] Thread[140025572345600] recompile single op[Conv_0] failed[FUNC:ProcessAllFailedCompileTasks][FILE:tbe_op_store_adapter.cc][LINE:954]
[SubGraphOpt][Compile][ProcFailedCompTask] Thread[140025572345600] recompile single op[Conv_6] failed[FUNC:ProcessAllFailedCompileTasks][FILE:tbe_op_store_adapter.cc][LINE:954]
[SubGraphOpt][Compile][ProcFailedCompTask] Thread[140025572345600] recompile single op[Conv_101] failed[FUNC:ProcessAllFailedCompileTasks][FILE:tbe_op_store_adapter.cc][LINE:954]
[SubGraphOpt][Compile][ProcFailedCompTask] Thread[140025572345600] recompile single op[Conv_90] failed[FUNC:ProcessAllFailedCompileTasks][FILE:tbe_op_store_adapter.cc][LINE:954]
[SubGraphOpt][Compile][ProcFailedCompTask] Thread[140025572345600] recompile single op[Conv_93] failed[FUNC:ProcessAllFailedCompileTasks][FILE:tbe_op_store_adapter.cc][LINE:954]
[SubGraphOpt][Compile][ProcFailedCompTask] Thread[140025572345600] recompile single op[Conv_47] failed[FUNC:ProcessAllFailedCompileTasks][FILE:tbe_op_store_adapter.cc][LINE:954]
[SubGraphOpt][Compile][ProcFailedCompTask] Thread[140025572345600] recompile single op[Conv_16] failed[FUNC:ProcessAllFailedCompileTasks][FILE:tbe_op_store_adapter.cc][LINE:954]
[SubGraphOpt][Compile][ProcFailedCompTask] Thread[140025572345600] recompile single op[Conv_32] failed[FUNC:ProcessAllFailedCompileTasks][FILE:tbe_op_store_adapter.cc][LINE:954]
[SubGraphOpt][Compile][ProcFailedCompTask] Thread[140025572345600] recompile single op[Conv_104] failed[FUNC:ProcessAllFailedCompileTasks][FILE:tbe_op_store_adapter.cc][LINE:954]
[SubGraphOpt][Compile][ProcFailedCompTask] Thread[140025572345600] recompile single op[Conv_39] failed[FUNC:ProcessAllFailedCompileTasks][FILE:tbe_op_store_adapter.cc][LINE:954]
[SubGraphOpt][Compile][ProcFailedCompTask] Thread[140025572345600] recompile single op[Conv_29] failed[FUNC:ProcessAllFailedCompileTasks][FILE:tbe_op_store_adapter.cc][LINE:954]
[SubGraphOpt][Compile][ProcFailedCompTask] Thread[140025572345600] recompile single op[Conv_87] failed[FUNC:ProcessAllFailedCompileTasks][FILE:tbe_op_store_adapter.cc][LINE:954]
[SubGraphOpt][Compile][ProcFailedCompTask] Thread[140025572345600] recompile single op[Conv_36] failed[FUNC:ProcessAllFailedCompileTasks][FILE:tbe_op_store_adapter.cc][LINE:954]
[SubGraphOpt][Compile][ProcFailedCompTask] Thread[140025572345600] recompile single op[Conv_97] failed[FUNC:ProcessAllFailedCompileTasks][FILE:tbe_op_store_adapter.cc][LINE:954]
[SubGraphOpt][Compile][ProcFailedCompTask] Thread[140025572345600] recompile single op[Conv_50] failed[FUNC:ProcessAllFailedCompileTasks][FILE:tbe_op_store_adapter.cc][LINE:954]
[SubGraphOpt][Compile][ProcFailedCompTask] Thread[140025572345600] recompile single op[Conv_111] failed[FUNC:ProcessAllFailedCompileTasks][FILE:tbe_op_store_adapter.cc][LINE:954]
[SubGraphOpt][Compile][ProcFailedCompTask] Thread[140025572345600] recompile single op[Conv_43] failed[FUNC:ProcessAllFailedCompileTasks][FILE:tbe_op_store_adapter.cc][LINE:954]
[SubGraphOpt][Compile][ProcFailedCompTask] Thread[140025572345600] recompile single op[Conv_26] failed[FUNC:ProcessAllFailedCompileTasks][FILE:tbe_op_store_adapter.cc][LINE:954]
[SubGraphOpt][Compile][ProcFailedCompTask] Thread[140025572345600] recompile single op[Conv_84] failed[FUNC:ProcessAllFailedCompileTasks][FILE:tbe_op_store_adapter.cc][LINE:954]
[SubGraphOpt][Compile][ProcFailedCompTask] Thread[140025572345600] recompile single op[Conv_77] failed[FUNC:ProcessAllFailedCompileTasks][FILE:tbe_op_store_adapter.cc][LINE:954]
[SubGraphOpt][Compile][ProcFailedCompTask] Thread[140025572345600] recompile single op[Conv_53] failed[FUNC:ProcessAllFailedCompileTasks][FILE:tbe_op_store_adapter.cc][LINE:954]
[SubGraphOpt][Compile][ProcFailedCompTask] Thread[140025572345600] recompile single op[Conv_59] failed[FUNC:ProcessAllFailedCompileTasks][FILE:tbe_op_store_adapter.cc][LINE:954]
[SubGraphOpt][Compile][ProcFailedCompTask] Thread[140025572345600] recompile single op[Conv_66] failed[FUNC:ProcessAllFailedCompileTasks][FILE:tbe_op_store_adapter.cc][LINE:954]
[SubGraphOpt][Compile][ProcFailedCompTask] Thread[140025572345600] recompile single op[Conv_73] failed[FUNC:ProcessAllFailedCompileTasks][FILE:tbe_op_store_adapter.cc][LINE:954]
[SubGraphOpt][Compile][ProcFailedCompTask] Thread[140025572345600] recompile single op[Conv_56] failed[FUNC:ProcessAllFailedCompileTasks][FILE:tbe_op_store_adapter.cc][LINE:954]
[SubGraphOpt][Compile][ProcFailedCompTask] Thread[140025572345600] recompile single op[Conv_23] failed[FUNC:ProcessAllFailedCompileTasks][FILE:tbe_op_store_adapter.cc][LINE:954]
[SubGraphOpt][Compile][ProcFailedCompTask] Thread[140025572345600] recompile single op[Conv_63] failed[FUNC:ProcessAllFailedCompileTasks][FILE:tbe_op_store_adapter.cc][LINE:954]
[SubGraphOpt][Compile][ProcFailedCompTask] Thread[140025572345600] recompile single op[Conv_9] failed[FUNC:ProcessAllFailedCompileTasks][FILE:tbe_op_store_adapter.cc][LINE:954]
[SubGraphOpt][Compile][ProcFailedCompTask] Thread[140025572345600] recompile single op[Conv_70] failed[FUNC:ProcessAllFailedCompileTasks][FILE:tbe_op_store_adapter.cc][LINE:954]
[SubGraphOpt][Compile][ProcFailedCompTask] Thread[140025572345600] recompile single op[Conv_114] failed[FUNC:ProcessAllFailedCompileTasks][FILE:tbe_op_store_adapter.cc][LINE:954]
[SubGraphOpt][Compile][ProcFailedCompTask] Thread[140025572345600] recompile single op[Conv_81] failed[FUNC:ProcessAllFailedCompileTasks][FILE:tbe_op_store_adapter.cc][LINE:954]
[SubGraphOpt][Compile][ParalCompOp] Thread[140025572345600] process fail task failed[FUNC:ParallelCompileOp][FILE:tbe_op_store_adapter.cc][LINE:1001]
Call OptimizeFusedGraph failed, ret:-1, engine_name:AIcoreEngine, graph_name:partition0_rank1_new_sub_graph1[FUNC:OptimizeSubGraph][FILE:graph_optimize.cc][LINE:131]
subgraph 0 optimize failed[FUNC:OptimizeSubGraphWithMultiThreads][FILE:graph_manager.cc][LINE:996]
build graph failed, graph id:0, ret:-1[FUNC:BuildModelWithGraphId][FILE:ge_generator.cc][LINE:1615]
GenerateOfflineModel execute failed.
Compile operator failed, cause: 32Broken pipe File "/home/lsn/Ascend/ascend-toolkit/latest/python/site-packages/tbe/common/repository_manager/interface.py", line 42, in cann_kb_finalize
RouteServer.finalize()
File "/home/lsn/Ascend/ascend-toolkit/latest/python/site-packages/tbe/common/repository_manager/route.py", line 51, in wrapper
return func(cls, *args, **kwargs)
File "/home/lsn/Ascend/ascend-toolkit/latest/python/site-packages/tbe/common/repository_manager/route.py", line 212, in finalize
cls.cann_kb_mgr.finalize()
File "/home/lsn/Ascend/ascend-toolkit/latest/python/site-packages/tbe/common/repository_manager/cann_kb_manager/knowledge_bank_manager.py", line 41, in finalize
self.__dynamic_mgr.close()
File "/home/lsn/Ascend/ascend-toolkit/latest/python/site-packages/tbe/common/repository_manager/cann_kb_manager/base_manager.py", line 327, in close
for name in copy.deepcopy(self.write_kb.keys()):
File "<string>", line 2, in keys
File "/usr/lib/python3.8/multiprocessing/managers.py", line 834, in _callmethod
conn.send((self._id, methodname, args, kwds))
File "/usr/lib/python3.8/multiprocessing/connection.py", line 206, in send
self._send_bytes(_ForkingPickler.dumps(obj))
File "/usr/lib/python3.8/multiprocessing/connection.py", line 411, in _send_bytes
self._send(header + buf)
File "/usr/lib/python3.8/multiprocessing/connection.py", line 368, in _send
n = write(self._handle, buf)
运行报错
lsn@lsn:~/infer_project$ atc --model=yolov5s.onnx --framework=5 --output=model --soc_version=Ascend310B4 ATC start working now, please wait for a moment. Exception in thread Thread-1: Traceback (most recent call last): File "/usr/lib/python3.8/threading.py", line 932, in _bootstrap_inner self.run() File "/home/lsn/Ascend/ascend-toolkit/latest/python/site-packages/tbe/common/repository_manager/utils/multiprocess_util.py", line 91, in run key, func, args, kwargs = self.task_q.get(timeout=TIMEOUT) File "<string>", line 2, in get File "/usr/lib/python3.8/multiprocessing/managers.py", line 835, in _callmethod kind, result = conn.recv() File "/usr/lib/python3.8/multiprocessing/connection.py", line 250, in recv buf = self._recv_bytes() File "/usr/lib/python3.8/multiprocessing/connection.py", line 414, in _recv_bytes buf = self._recv(4) File "/usr/lib/python3.8/multiprocessing/connection.py", line 383, in _recv raise EOFError EOFError Traceback (most recent call last): File "/usr/lib/python3.8/multiprocessing/util.py", line 300, in _run_finalizers finalizer() File "/usr/lib/python3.8/multiprocessing/util.py", line 224, in __call__ res = self._callback(*self._args, **self._kwargs) File "/home/lsn/Ascend/ascend-toolkit/latest/python/site-packages/tbe/common/repository_manager/route.py", line 51, in wrapper return func(cls, *args, **kwargs) File "/home/lsn/Ascend/ascend-toolkit/latest/python/site-packages/tbe/common/repository_manager/route.py", line 207, in finalize cls.bank_process_pool.close() AttributeError: 'NoneType' object has no attribute 'close' ... ATC run failed, Please check the detail log, Try 'atc --help' for more information E40021: Failed to compile Op [Conv_272,[Conv_272,Conv_272]]. (oppath: [Compile /home/lsn/Ascend/ascend-toolkit/7.0.1/opp/built-in/op_impl/ai_core/tbe/impl/conv2d.py failed with errormsg/stack: File "/usr/lib/python3.8/multiprocessing/connection.py", line 630, in SocketClient s.connect(address) ConnectionRefusedError: [Errno 111] Connection refused ], optype: [Conv2D]) Solution: See the host log for details, and then check the Python stack where the error log is reported. TraceBack (most recent call last): Compile op[Conv_272,[Conv_272,Conv_272]] failed, oppath[/home/lsn/Ascend/ascend-toolkit/7.0.1/opp/built-in/op_impl/ai_core/tbe/impl/conv2d.py], optype[Conv2D], taskID[270]. Please check op's compilation error message.[FUNC:ReportBuildErrMessage][FILE:fusion_manager.cc][LINE:771] Failed to compile Op [Conv_235,[Conv_235,Conv_235]]. (oppath: [Compile /home/lsn/Ascend/ascend-toolkit/7.0.1/opp/built-in/op_impl/ai_core/tbe/impl/conv2d.py failed with errormsg/stack: File "/usr/lib/python3.8/multiprocessing/connection.py", line 630, in SocketClient s.connect(address) ConnectionRefusedError: [Errno 111] Connection refused ], optype: [Conv2D]) Compile op[Conv_235,[Conv_235,Conv_235]] failed, oppath[/home/lsn/Ascend/ascend-toolkit/7.0.1/opp/built-in/op_impl/ai_core/tbe/impl/conv2d.py], optype[Conv2D], taskID[274]. Please check op's compilation error message.[FUNC:ReportBuildErrMessage][FILE:fusion_manager.cc][LINE:771] Failed to compile Op [Conv_198,[Conv_198,Conv_198]]. (oppath: [Compile /home/lsn/Ascend/ascend-toolkit/7.0.1/opp/built-in/op_impl/ai_core/tbe/impl/conv2d.py failed with errormsg/stack: File "/usr/lib/python3.8/multiprocessing/connection.py", line 630, in SocketClient s.connect(address) ConnectionRefusedError: [Errno 111] Connection refused ], optype: [Conv2D]) Compile op[Conv_198,[Conv_198,Conv_198]] failed, oppath[/home/lsn/Ascend/ascend-toolkit/7.0.1/opp/built-in/op_impl/ai_core/tbe/impl/conv2d.py], optype[Conv2D], taskID[278]. Please check op's compilation error message.[FUNC:ReportBuildErrMessage][FILE:fusion_manager.cc][LINE:771] Failed to compile Op [Conv_133,[Conv_133,Conv_133]]. (oppath: [Compile /home/lsn/Ascend/ascend-toolkit/7.0.1/opp/built-in/op_impl/ai_core/tbe/impl/conv2d.py failed with errormsg/stack: File "/usr/lib/python3.8/multiprocessing/connection.py", line 630, in SocketClient s.connect(address) ConnectionRefusedError: [Errno 111] Connection refused ], optype: [Conv2D]) Compile op[Conv_133,[Conv_133,Conv_133]] failed, oppath[/home/lsn/Ascend/ascend-toolkit/7.0.1/opp/built-in/op_impl/ai_core/tbe/impl/conv2d.py], optype[Conv2D], taskID[350]. Please check op's compilation error message.[FUNC:ReportBuildErrMessage][FILE:fusion_manager.cc][LINE:771] Failed to compile Op [Conv_123,[Conv_123,Conv_123]]. (oppath: [Compile /home/lsn/Ascend/ascend-toolkit/7.0.1/opp/built-in/op_impl/ai_core/tbe/impl/conv2d.py failed with errormsg/stack: File "/usr/lib/python3.8/multiprocessing/connection.py", line 630, in SocketClient s.connect(address) ConnectionRefusedError: [Errno 111] Connection refused ], optype: [Conv2D]) Compile op[Conv_123,[Conv_123,Conv_123]] failed, oppath[/home/lsn/Ascend/ascend-toolkit/7.0.1/opp/built-in/op_impl/ai_core/tbe/impl/conv2d.py], optype[Conv2D], taskID[353]. Please check op's compilation error message.[FUNC:ReportBuildErrMessage][FILE:fusion_manager.cc][LINE:771] Failed to compile Op [Conv_126,[Conv_126,Conv_126]]. (oppath: [Compile /home/lsn/Ascend/ascend-toolkit/7.0.1/opp/built-in/op_impl/ai_core/tbe/impl/conv2d.py failed with errormsg/stack: File "/usr/lib/python3.8/multiprocessing/connection.py", line 630, in SocketClient s.connect(address) ConnectionRefusedError: [Errno 111] Connection refused ], optype: [Conv2D]) Compile op[Conv_126,[Conv_126,Conv_126]] failed, oppath[/home/lsn/Ascend/ascend-toolkit/7.0.1/opp/built-in/op_impl/ai_core/tbe/impl/conv2d.py], optype[Conv2D], taskID[356]. Please check op's compilation error message.[FUNC:ReportBuildErrMessage][FILE:fusion_manager.cc][LINE:771] Failed to compile Op [Conv_136,[Conv_136,Conv_136]]. (oppath: [Compile /home/lsn/Ascend/ascend-toolkit/7.0.1/opp/built-in/op_impl/ai_core/tbe/impl/conv2d.py failed with errormsg/stack: File "/usr/lib/python3.8/multiprocessing/connection.py", line 630, in SocketClient s.connect(address) ConnectionRefusedError: [Errno 111] Connection refused ], optype: [Conv2D]) Compile op[Conv_136,[Conv_136,Conv_136]] failed, oppath[/home/lsn/Ascend/ascend-toolkit/7.0.1/opp/built-in/op_impl/ai_core/tbe/impl/conv2d.py], optype[Conv2D], taskID[365]. Please check op's compilation error message.[FUNC:ReportBuildErrMessage][FILE:fusion_manager.cc][LINE:771] Failed to compile Op [Conv_120,[Conv_120,Conv_120]]. (oppath: [Compile /home/lsn/Ascend/ascend-toolkit/7.0.1/opp/built-in/op_impl/ai_core/tbe/impl/conv2d.py failed with errormsg/stack: File "/usr/lib/python3.8/multiprocessing/connection.py", line 630, in SocketClient s.connect(address) ConnectionRefusedError: [Errno 111] Connection refused ], optype: [Conv2D]) Compile op[Conv_120,[Conv_120,Conv_120]] failed, oppath[/home/lsn/Ascend/ascend-toolkit/7.0.1/opp/built-in/op_impl/ai_core/tbe/impl/conv2d.py], optype[Conv2D], taskID[359]. Please check op's compilation error message.[FUNC:ReportBuildErrMessage][FILE:fusion_manager.cc][LINE:771] Failed to compile Op [Conv_84,[Conv_84,Conv_84]]. (oppath: [Compile /home/lsn/Ascend/ascend-toolkit/7.0.1/opp/built-in/op_impl/ai_core/tbe/impl/conv2d.py failed with errormsg/stack: File "/usr/lib/python3.8/multiprocessing/connection.py", line 630, in SocketClient s.connect(address) ConnectionRefusedError: [Errno 111] Connection refused ], optype: [Conv2D]) Compile op[Conv_84,[Conv_84,Conv_84]] failed, oppath[/home/lsn/Ascend/ascend-toolkit/7.0.1/opp/built-in/op_impl/ai_core/tbe/impl/conv2d.py], optype[Conv2D], taskID[368]. Please check op's compilation error message.[FUNC:ReportBuildErrMessage][FILE:fusion_manager.cc][LINE:771] Failed to compile Op [Conv_26,[Conv_26,Conv_26]]. (oppath: [Compile /home/lsn/Ascend/ascend-toolkit/7.0.1/opp/built-in/op_impl/ai_core/tbe/impl/conv2d.py failed with errormsg/stack: File "/usr/lib/python3.8/multiprocessing/connection.py", line 630, in SocketClient s.connect(address) ConnectionRefusedError: [Errno 111] Connection refused ], optype: [Conv2D]) Compile op[Conv_26,[Conv_26,Conv_26]] failed, oppath[/home/lsn/Ascend/ascend-toolkit/7.0.1/opp/built-in/op_impl/ai_core/tbe/impl/conv2d.py], optype[Conv2D], taskID[371]. Please check op's compilation error message.[FUNC:ReportBuildErrMessage][FILE:fusion_manager.cc][LINE:771] Failed to compile Op [Conv_111,[Conv_111,Conv_111]]. (oppath: [Compile /home/lsn/Ascend/ascend-toolkit/7.0.1/opp/built-in/op_impl/ai_core/tbe/impl/conv2d.py failed with errormsg/stack: File "/usr/lib/python3.8/multiprocessing/connection.py", line 630, in SocketClient s.connect(address) ConnectionRefusedError: [Errno 111] Connection refused ], optype: [Conv2D]) Compile op[Conv_111,[Conv_111,Conv_111]] failed, oppath[/home/lsn/Ascend/ascend-toolkit/7.0.1/opp/built-in/op_impl/ai_core/tbe/impl/conv2d.py], optype[Conv2D], taskID[377]. Please check op's compilation error message.[FUNC:ReportBuildErrMessage][FILE:fusion_manager.cc][LINE:771] Failed to compile Op [Conv_50,[Conv_50,Conv_50]]. (oppath: [Compile /home/lsn/Ascend/ascend-toolkit/7.0.1/opp/built-in/op_impl/ai_core/tbe/impl/conv2d.py failed with errormsg/stack: File "/usr/lib/python3.8/multiprocessing/connection.py", line 630, in SocketClient s.connect(address) ConnectionRefusedError: [Errno 111] Connection refused ], optype: [Conv2D]) Compile op[Conv_50,[Conv_50,Conv_50]] failed, oppath[/home/lsn/Ascend/ascend-toolkit/7.0.1/opp/built-in/op_impl/ai_core/tbe/impl/conv2d.py], optype[Conv2D], taskID[380]. Please check op's compilation error message.[FUNC:ReportBuildErrMessage][FILE:fusion_manager.cc][LINE:771] Failed to compile Op [Conv_36,[Conv_36,Conv_36]]. (oppath: [Compile /home/lsn/Ascend/ascend-toolkit/7.0.1/opp/built-in/op_impl/ai_core/tbe/impl/conv2d.py failed with errormsg/stack: File "/usr/lib/python3.8/multiprocessing/connection.py", line 630, in SocketClient s.connect(address) ConnectionRefusedError: [Errno 111] Connection refused ], optype: [Conv2D]) Compile op[Conv_36,[Conv_36,Conv_36]] failed, oppath[/home/lsn/Ascend/ascend-toolkit/7.0.1/opp/built-in/op_impl/ai_core/tbe/impl/conv2d.py], optype[Conv2D], taskID[383]. Please check op's compilation error message.[FUNC:ReportBuildErrMessage][FILE:fusion_manager.cc][LINE:771] Failed to compile Op [Conv_39,[Conv_39,Conv_39]]. (oppath: [Compile /home/lsn/Ascend/ascend-toolkit/7.0.1/opp/built-in/op_impl/ai_core/tbe/impl/conv2d.py failed with errormsg/stack: File "/usr/lib/python3.8/multiprocessing/connection.py", line 630, in SocketClient s.connect(address) ConnectionRefusedError: [Errno 111] Connection refused ], optype: [Conv2D]) Compile op[Conv_39,[Conv_39,Conv_39]] failed, oppath[/home/lsn/Ascend/ascend-toolkit/7.0.1/opp/built-in/op_impl/ai_core/tbe/impl/conv2d.py], optype[Conv2D], taskID[386]. Please check op's compilation error message.[FUNC:ReportBuildErrMessage][FILE:fusion_manager.cc][LINE:771] [SubGraphOpt][Compile][ProcFailedCompTask] Thread[140026226648832] recompile single op[Conv_129] failed[FUNC:ProcessAllFailedCompileTasks][FILE:tbe_op_store_adapter.cc][LINE:954] [SubGraphOpt][Compile][ProcFailedCompTask] Thread[140026226648832] recompile single op[Conv_120] failed[FUNC:ProcessAllFailedCompileTasks][FILE:tbe_op_store_adapter.cc][LINE:954] [SubGraphOpt][Compile][ProcFailedCompTask] Thread[140026226648832] recompile single op[Conv_136] failed[FUNC:ProcessAllFailedCompileTasks][FILE:tbe_op_store_adapter.cc][LINE:954] [SubGraphOpt][Compile][ProcFailedCompTask] Thread[140026226648832] recompile single op[Conv_126] failed[FUNC:ProcessAllFailedCompileTasks][FILE:tbe_op_store_adapter.cc][LINE:954] [SubGraphOpt][Compile][ProcFailedCompTask] Thread[140026226648832] recompile single op[Conv_123] failed[FUNC:ProcessAllFailedCompileTasks][FILE:tbe_op_store_adapter.cc][LINE:954] [SubGraphOpt][Compile][ProcFailedCompTask] Thread[140026226648832] recompile single op[Conv_133] failed[FUNC:ProcessAllFailedCompileTasks][FILE:tbe_op_store_adapter.cc][LINE:954] [SubGraphOpt][Compile][ParalCompOp] Thread[140026226648832] process fail task failed[FUNC:ParallelCompileOp][FILE:tbe_op_store_adapter.cc][LINE:1001] [SubGraphOpt][Compile][CompOpOnly] CompileOp failed.[FUNC:CompileOpOnly][FILE:op_compiler.cc][LINE:1127] [GraphOpt][FusedGraph][RunCompile] Failed to compile graph with compiler Normal mode Op Compiler[FUNC:SubGraphCompile][FILE:fe_graph_optimizer.cc][LINE:1292] Call OptimizeFusedGraph failed, ret:-1, engine_name:AIcoreEngine, graph_name:partition0_rank3_new_sub_graph133[FUNC:OptimizeSubGraph][FILE:graph_optimize.cc][LINE:131] Failed to compile Op [Conv_47,[Conv_47,Conv_47]]. (oppath: [Compile /home/lsn/Ascend/ascend-toolkit/7.0.1/opp/built-in/op_impl/ai_core/tbe/impl/conv2d.py failed with errormsg/stack: File "/usr/lib/python3.8/multiprocessing/connection.py", line 630, in SocketClient s.connect(address) ConnectionRefusedError: [Errno 111] Connection refused ], optype: [Conv2D]) Compile op[Conv_47,[Conv_47,Conv_47]] failed, oppath[/home/lsn/Ascend/ascend-toolkit/7.0.1/opp/built-in/op_impl/ai_core/tbe/impl/conv2d.py], optype[Conv2D], taskID[422]. Please check op's compilation error message.[FUNC:ReportBuildErrMessage][FILE:fusion_manager.cc][LINE:771] Failed to compile Op [Conv_188,[Conv_188,Conv_188]]. (oppath: [Compile /home/lsn/Ascend/ascend-toolkit/7.0.1/opp/built-in/op_impl/ai_core/tbe/impl/conv2d.py failed with errormsg/stack: File "/usr/lib/python3.8/multiprocessing/connection.py", line 630, in SocketClient s.connect(address) ConnectionRefusedError: [Errno 111] Connection refused ], optype: [Conv2D]) Compile op[Conv_188,[Conv_188,Conv_188]] failed, oppath[/home/lsn/Ascend/ascend-toolkit/7.0.1/opp/built-in/op_impl/ai_core/tbe/impl/conv2d.py], optype[Conv2D], taskID[428]. Please check op's compilation error message.[FUNC:ReportBuildErrMessage][FILE:fusion_manager.cc][LINE:771] Failed to compile Op [Conv_185,[Conv_185,Conv_185]]. (oppath: [Compile /home/lsn/Ascend/ascend-toolkit/7.0.1/opp/built-in/op_impl/ai_core/tbe/impl/conv2d.py failed with errormsg/stack: File "/usr/lib/python3.8/multiprocessing/connection.py", line 630, in SocketClient s.connect(address) ConnectionRefusedError: [Errno 111] Connection refused ], optype: [Conv2D]) Compile op[Conv_185,[Conv_185,Conv_185]] failed, oppath[/home/lsn/Ascend/ascend-toolkit/7.0.1/opp/built-in/op_impl/ai_core/tbe/impl/conv2d.py], optype[Conv2D], taskID[431]. Please check op's compilation error message.[FUNC:ReportBuildErrMessage][FILE:fusion_manager.cc][LINE:771] Failed to compile Op [Conv_195,[Conv_195,Conv_195]]. (oppath: [Compile /home/lsn/Ascend/ascend-toolkit/7.0.1/opp/built-in/op_impl/ai_core/tbe/impl/conv2d.py failed with errormsg/stack: File "/usr/lib/python3.8/multiprocessing/connection.py", line 630, in SocketClient s.connect(address) ConnectionRefusedError: [Errno 111] Connection refused ], optype: [Conv2D]) Compile op[Conv_195,[Conv_195,Conv_195]] failed, oppath[/home/lsn/Ascend/ascend-toolkit/7.0.1/opp/built-in/op_impl/ai_core/tbe/impl/conv2d.py], optype[Conv2D], taskID[425]. Please check op's compilation error message.[FUNC:ReportBuildErrMessage][FILE:fusion_manager.cc][LINE:771] Failed to compile Op [Conv_178,[Conv_178,Conv_178]]. (oppath: [Compile /home/lsn/Ascend/ascend-toolkit/7.0.1/opp/built-in/op_impl/ai_core/tbe/impl/conv2d.py failed with errormsg/stack: File "/usr/lib/python3.8/multiprocessing/connection.py", line 630, in SocketClient s.connect(address) ConnectionRefusedError: [Errno 111] Connection refused ], optype: [Conv2D]) Compile op[Conv_178,[Conv_178,Conv_178]] failed, oppath[/home/lsn/Ascend/ascend-toolkit/7.0.1/opp/built-in/op_impl/ai_core/tbe/impl/conv2d.py], optype[Conv2D], taskID[434]. Please check op's compilation error message.[FUNC:ReportBuildErrMessage][FILE:fusion_manager.cc][LINE:771] Failed to compile Op [Conv_182,[Conv_182,Conv_182]]. (oppath: [Compile /home/lsn/Ascend/ascend-toolkit/7.0.1/opp/built-in/op_impl/ai_core/tbe/impl/conv2d.py failed with errormsg/stack: File "/usr/lib/python3.8/multiprocessing/connection.py", line 630, in SocketClient s.connect(address) ConnectionRefusedError: [Errno 111] Connection refused ], optype: [Conv2D]) Compile op[Conv_182,[Conv_182,Conv_182]] failed, oppath[/home/lsn/Ascend/ascend-toolkit/7.0.1/opp/built-in/op_impl/ai_core/tbe/impl/conv2d.py], optype[Conv2D], taskID[437]. Please check op's compilation error message.[FUNC:ReportBuildErrMessage][FILE:fusion_manager.cc][LINE:771] Failed to compile Op [Conv_16,[Conv_16,Conv_16]]. (oppath: [Compile /home/lsn/Ascend/ascend-toolkit/7.0.1/opp/built-in/op_impl/ai_core/tbe/impl/conv2d.py failed with errormsg/stack: File "/usr/lib/python3.8/multiprocessing/connection.py", line 630, in SocketClient s.connect(address) ConnectionRefusedError: [Errno 111] Connection refused ], optype: [Conv2D]) Compile op[Conv_16,[Conv_16,Conv_16]] failed, oppath[/home/lsn/Ascend/ascend-toolkit/7.0.1/opp/built-in/op_impl/ai_core/tbe/impl/conv2d.py], optype[Conv2D], taskID[448]. Please check op's compilation error message.[FUNC:ReportBuildErrMessage][FILE:fusion_manager.cc][LINE:771] Failed to compile Op [Conv_9,[Conv_9,Conv_9]]. (oppath: [Compile /home/lsn/Ascend/ascend-toolkit/7.0.1/opp/built-in/op_impl/ai_core/tbe/impl/conv2d.py failed with errormsg/stack: File "/usr/lib/python3.8/multiprocessing/connection.py", line 630, in SocketClient s.connect(address) ConnectionRefusedError: [Errno 111] Connection refused ], optype: [Conv2D]) Compile op[Conv_9,[Conv_9,Conv_9]] failed, oppath[/home/lsn/Ascend/ascend-toolkit/7.0.1/opp/built-in/op_impl/ai_core/tbe/impl/conv2d.py], optype[Conv2D], taskID[454]. Please check op's compilation error message.[FUNC:ReportBuildErrMessage][FILE:fusion_manager.cc][LINE:771] Failed to compile Op [Conv_0,[Conv_0,Conv_0]]. (oppath: [Compile /home/lsn/Ascend/ascend-toolkit/7.0.1/opp/built-in/op_impl/ai_core/tbe/impl/conv2d.py failed with errormsg/stack: File "/usr/lib/python3.8/multiprocessing/connection.py", line 630, in SocketClient s.connect(address) ConnectionRefusedError: [Errno 111] Connection refused ], optype: [Conv2D]) Compile op[Conv_0,[Conv_0,Conv_0]] failed, oppath[/home/lsn/Ascend/ascend-toolkit/7.0.1/opp/built-in/op_impl/ai_core/tbe/impl/conv2d.py], optype[Conv2D], taskID[464]. Please check op's compilation error message.[FUNC:ReportBuildErrMessage][FILE:fusion_manager.cc][LINE:771] Failed to compile Op [Conv_158,[Conv_158,Conv_158]]. (oppath: [Compile /home/lsn/Ascend/ascend-toolkit/7.0.1/opp/built-in/op_impl/ai_core/tbe/impl/conv2d.py failed with errormsg/stack: File "/usr/lib/python3.8/multiprocessing/connection.py", line 630, in SocketClient s.connect(address) ConnectionRefusedError: [Errno 111] Connection refused ], optype: [Conv2D]) Compile op[Conv_158,[Conv_158,Conv_158]] failed, oppath[/home/lsn/Ascend/ascend-toolkit/7.0.1/opp/built-in/op_impl/ai_core/tbe/impl/conv2d.py], optype[Conv2D], taskID[471]. Please check op's compilation error message.[FUNC:ReportBuildErrMessage][FILE:fusion_manager.cc][LINE:771] Failed to compile Op [Conv_12,[Conv_12,Conv_12]]. (oppath: [Compile /home/lsn/Ascend/ascend-toolkit/7.0.1/opp/built-in/op_impl/ai_core/tbe/impl/conv2d.py failed with errormsg/stack: File "/usr/lib/python3.8/multiprocessing/connection.py", line 630, in SocketClient s.connect(address) ConnectionRefusedError: [Errno 111] Connection refused ], optype: [Conv2D]) Compile op[Conv_12,[Conv_12,Conv_12]] failed, oppath[/home/lsn/Ascend/ascend-toolkit/7.0.1/opp/built-in/op_impl/ai_core/tbe/impl/conv2d.py], optype[Conv2D], taskID[496]. Please check op's compilation error message.[FUNC:ReportBuildErrMessage][FILE:fusion_manager.cc][LINE:771] Failed to compile Op [Conv_3,[Conv_3,Conv_3]]. (oppath: [Compile /home/lsn/Ascend/ascend-toolkit/7.0.1/opp/built-in/op_impl/ai_core/tbe/impl/conv2d.py failed with errormsg/stack: File "/usr/lib/python3.8/multiprocessing/connection.py", line 630, in SocketClient s.connect(address) ConnectionRefusedError: [Errno 111] Connection refused ], optype: [Conv2D]) Compile op[Conv_3,[Conv_3,Conv_3]] failed, oppath[/home/lsn/Ascend/ascend-toolkit/7.0.1/opp/built-in/op_impl/ai_core/tbe/impl/conv2d.py], optype[Conv2D], taskID[490]. Please check op's compilation error message.[FUNC:ReportBuildErrMessage][FILE:fusion_manager.cc][LINE:771] Failed to compile Op [Conv_20,[Conv_20,Conv_20]]. (oppath: [Compile /home/lsn/Ascend/ascend-toolkit/7.0.1/opp/built-in/op_impl/ai_core/tbe/impl/conv2d.py failed with errormsg/stack: File "/usr/lib/python3.8/multiprocessing/connection.py", line 630, in SocketClient s.connect(address) ConnectionRefusedError: [Errno 111] Connection refused ], optype: [Conv2D]) Compile op[Conv_20,[Conv_20,Conv_20]] failed, oppath[/home/lsn/Ascend/ascend-toolkit/7.0.1/opp/built-in/op_impl/ai_core/tbe/impl/conv2d.py], optype[Conv2D], taskID[499]. Please check op's compilation error message.[FUNC:ReportBuildErrMessage][FILE:fusion_manager.cc][LINE:771] Failed to compile Op [Conv_142,[Conv_142,Conv_142]]. (oppath: [Compile /home/lsn/Ascend/ascend-toolkit/7.0.1/opp/built-in/op_impl/ai_core/tbe/impl/conv2d.py failed with errormsg/stack: File "/usr/lib/python3.8/multiprocessing/connection.py", line 630, in SocketClient s.connect(address) ConnectionRefusedError: [Errno 111] Connection refused ], optype: [Conv2D]) Compile op[Conv_142,[Conv_142,Conv_142]] failed, oppath[/home/lsn/Ascend/ascend-toolkit/7.0.1/opp/built-in/op_impl/ai_core/tbe/impl/conv2d.py], optype[Conv2D], taskID[504]. Please check op's compilation error message.[FUNC:ReportBuildErrMessage][FILE:fusion_manager.cc][LINE:771] [SubGraphOpt][Compile][ProcFailedCompTask] Thread[140026251826944] recompile single op[Conv_151] failed[FUNC:ProcessAllFailedCompileTasks][FILE:tbe_op_store_adapter.cc][LINE:954] [SubGraphOpt][Compile][ProcFailedCompTask] Thread[140026251826944] recompile single op[Conv_142] failed[FUNC:ProcessAllFailedCompileTasks][FILE:tbe_op_store_adapter.cc][LINE:954] [SubGraphOpt][Compile][ProcFailedCompTask] Thread[140026251826944] recompile single op[Conv_158] failed[FUNC:ProcessAllFailedCompileTasks][FILE:tbe_op_store_adapter.cc][LINE:954] [SubGraphOpt][Compile][ProcFailedCompTask] Thread[140026251826944] recompile single op[Conv_191] failed[FUNC:ProcessAllFailedCompileTasks][FILE:tbe_op_store_adapter.cc][LINE:954] [SubGraphOpt][Compile][ProcFailedCompTask] Thread[140026251826944] recompile single op[Conv_182] failed[FUNC:ProcessAllFailedCompileTasks][FILE:tbe_op_store_adapter.cc][LINE:954] [SubGraphOpt][Compile][ProcFailedCompTask] Thread[140026251826944] recompile single op[Conv_185] failed[FUNC:ProcessAllFailedCompileTasks][FILE:tbe_op_store_adapter.cc][LINE:954] [SubGraphOpt][Compile][ProcFailedCompTask] Thread[140026251826944] recompile single op[Conv_188] failed[FUNC:ProcessAllFailedCompileTasks][FILE:tbe_op_store_adapter.cc][LINE:954] [SubGraphOpt][Compile][ProcFailedCompTask] Thread[140026251826944] recompile single op[Conv_178] failed[FUNC:ProcessAllFailedCompileTasks][FILE:tbe_op_store_adapter.cc][LINE:954] [SubGraphOpt][Compile][ProcFailedCompTask] Thread[140026251826944] recompile single op[Conv_155] failed[FUNC:ProcessAllFailedCompileTasks][FILE:tbe_op_store_adapter.cc][LINE:954] [SubGraphOpt][Compile][ProcFailedCompTask] Thread[140026251826944] recompile single op[Conv_148] failed[FUNC:ProcessAllFailedCompileTasks][FILE:tbe_op_store_adapter.cc][LINE:954] [SubGraphOpt][Compile][ProcFailedCompTask] Thread[140026251826944] recompile single op[Conv_145] failed[FUNC:ProcessAllFailedCompileTasks][FILE:tbe_op_store_adapter.cc][LINE:954] [SubGraphOpt][Compile][ProcFailedCompTask] Thread[140026251826944] recompile single op[Conv_235] failed[FUNC:ProcessAllFailedCompileTasks][FILE:tbe_op_store_adapter.cc][LINE:954] [SubGraphOpt][Compile][ProcFailedCompTask] Thread[140026251826944] recompile single op[Conv_272] failed[FUNC:ProcessAllFailedCompileTasks][FILE:tbe_op_store_adapter.cc][LINE:954] [SubGraphOpt][Compile][ProcFailedCompTask] Thread[140026251826944] recompile single op[Conv_175] failed[FUNC:ProcessAllFailedCompileTasks][FILE:tbe_op_store_adapter.cc][LINE:954] [SubGraphOpt][Compile][ProcFailedCompTask] Thread[140026251826944] recompile single op[Conv_165] failed[FUNC:ProcessAllFailedCompileTasks][FILE:tbe_op_store_adapter.cc][LINE:954] [SubGraphOpt][Compile][ProcFailedCompTask] Thread[140026251826944] recompile single op[Conv_195] failed[FUNC:ProcessAllFailedCompileTasks][FILE:tbe_op_store_adapter.cc][LINE:954] [SubGraphOpt][Compile][ProcFailedCompTask] Thread[140026251826944] recompile single op[Conv_198] failed[FUNC:ProcessAllFailedCompileTasks][FILE:tbe_op_store_adapter.cc][LINE:954] [SubGraphOpt][Compile][ProcFailedCompTask] Thread[140026251826944] recompile single op[Conv_168] failed[FUNC:ProcessAllFailedCompileTasks][FILE:tbe_op_store_adapter.cc][LINE:954] [SubGraphOpt][Compile][ProcFailedCompTask] Thread[140026251826944] recompile single op[Conv_162] failed[FUNC:ProcessAllFailedCompileTasks][FILE:tbe_op_store_adapter.cc][LINE:954] [SubGraphOpt][Compile][ProcFailedCompTask] Thread[140026251826944] recompile single op[Conv_171] failed[FUNC:ProcessAllFailedCompileTasks][FILE:tbe_op_store_adapter.cc][LINE:954] [SubGraphOpt][Compile][ParalCompOp] Thread[140026251826944] process fail task failed[FUNC:ParallelCompileOp][FILE:tbe_op_store_adapter.cc][LINE:1001] Call OptimizeFusedGraph failed, ret:-1, engine_name:AIcoreEngine, graph_name:partition0_rank5_new_sub_graph135[FUNC:OptimizeSubGraph][FILE:graph_optimize.cc][LINE:131] Failed to compile Op [Conv_23,[Conv_23,Conv_23]]. (oppath: [Compile /home/lsn/Ascend/ascend-toolkit/7.0.1/opp/built-in/op_impl/ai_core/tbe/impl/conv2d.py failed with errormsg/stack: File "/usr/lib/python3.8/multiprocessing/connection.py", line 630, in SocketClient s.connect(address) ConnectionRefusedError: [Errno 111] Connection refused ], optype: [Conv2D]) Compile op[Conv_23,[Conv_23,Conv_23]] failed, oppath[/home/lsn/Ascend/ascend-toolkit/7.0.1/opp/built-in/op_impl/ai_core/tbe/impl/conv2d.py], optype[Conv2D], taskID[516]. Please check op's compilation error message.[FUNC:ReportBuildErrMessage][FILE:fusion_manager.cc][LINE:771] [SubGraphOpt][Compile][ProcFailedCompTask] Thread[140025572345600] recompile single op[Conv_20] failed[FUNC:ProcessAllFailedCompileTasks][FILE:tbe_op_store_adapter.cc][LINE:954] [SubGraphOpt][Compile][ProcFailedCompTask] Thread[140025572345600] recompile single op[Conv_3] failed[FUNC:ProcessAllFailedCompileTasks][FILE:tbe_op_store_adapter.cc][LINE:954] [SubGraphOpt][Compile][ProcFailedCompTask] Thread[140025572345600] recompile single op[Conv_12] failed[FUNC:ProcessAllFailedCompileTasks][FILE:tbe_op_store_adapter.cc][LINE:954] [SubGraphOpt][Compile][ProcFailedCompTask] Thread[140025572345600] recompile single op[Conv_0] failed[FUNC:ProcessAllFailedCompileTasks][FILE:tbe_op_store_adapter.cc][LINE:954] [SubGraphOpt][Compile][ProcFailedCompTask] Thread[140025572345600] recompile single op[Conv_6] failed[FUNC:ProcessAllFailedCompileTasks][FILE:tbe_op_store_adapter.cc][LINE:954] [SubGraphOpt][Compile][ProcFailedCompTask] Thread[140025572345600] recompile single op[Conv_101] failed[FUNC:ProcessAllFailedCompileTasks][FILE:tbe_op_store_adapter.cc][LINE:954] [SubGraphOpt][Compile][ProcFailedCompTask] Thread[140025572345600] recompile single op[Conv_90] failed[FUNC:ProcessAllFailedCompileTasks][FILE:tbe_op_store_adapter.cc][LINE:954] [SubGraphOpt][Compile][ProcFailedCompTask] Thread[140025572345600] recompile single op[Conv_93] failed[FUNC:ProcessAllFailedCompileTasks][FILE:tbe_op_store_adapter.cc][LINE:954] [SubGraphOpt][Compile][ProcFailedCompTask] Thread[140025572345600] recompile single op[Conv_47] failed[FUNC:ProcessAllFailedCompileTasks][FILE:tbe_op_store_adapter.cc][LINE:954] [SubGraphOpt][Compile][ProcFailedCompTask] Thread[140025572345600] recompile single op[Conv_16] failed[FUNC:ProcessAllFailedCompileTasks][FILE:tbe_op_store_adapter.cc][LINE:954] [SubGraphOpt][Compile][ProcFailedCompTask] Thread[140025572345600] recompile single op[Conv_32] failed[FUNC:ProcessAllFailedCompileTasks][FILE:tbe_op_store_adapter.cc][LINE:954] [SubGraphOpt][Compile][ProcFailedCompTask] Thread[140025572345600] recompile single op[Conv_104] failed[FUNC:ProcessAllFailedCompileTasks][FILE:tbe_op_store_adapter.cc][LINE:954] [SubGraphOpt][Compile][ProcFailedCompTask] Thread[140025572345600] recompile single op[Conv_39] failed[FUNC:ProcessAllFailedCompileTasks][FILE:tbe_op_store_adapter.cc][LINE:954] [SubGraphOpt][Compile][ProcFailedCompTask] Thread[140025572345600] recompile single op[Conv_29] failed[FUNC:ProcessAllFailedCompileTasks][FILE:tbe_op_store_adapter.cc][LINE:954] [SubGraphOpt][Compile][ProcFailedCompTask] Thread[140025572345600] recompile single op[Conv_87] failed[FUNC:ProcessAllFailedCompileTasks][FILE:tbe_op_store_adapter.cc][LINE:954] [SubGraphOpt][Compile][ProcFailedCompTask] Thread[140025572345600] recompile single op[Conv_36] failed[FUNC:ProcessAllFailedCompileTasks][FILE:tbe_op_store_adapter.cc][LINE:954] [SubGraphOpt][Compile][ProcFailedCompTask] Thread[140025572345600] recompile single op[Conv_97] failed[FUNC:ProcessAllFailedCompileTasks][FILE:tbe_op_store_adapter.cc][LINE:954] [SubGraphOpt][Compile][ProcFailedCompTask] Thread[140025572345600] recompile single op[Conv_50] failed[FUNC:ProcessAllFailedCompileTasks][FILE:tbe_op_store_adapter.cc][LINE:954] [SubGraphOpt][Compile][ProcFailedCompTask] Thread[140025572345600] recompile single op[Conv_111] failed[FUNC:ProcessAllFailedCompileTasks][FILE:tbe_op_store_adapter.cc][LINE:954] [SubGraphOpt][Compile][ProcFailedCompTask] Thread[140025572345600] recompile single op[Conv_43] failed[FUNC:ProcessAllFailedCompileTasks][FILE:tbe_op_store_adapter.cc][LINE:954] [SubGraphOpt][Compile][ProcFailedCompTask] Thread[140025572345600] recompile single op[Conv_26] failed[FUNC:ProcessAllFailedCompileTasks][FILE:tbe_op_store_adapter.cc][LINE:954] [SubGraphOpt][Compile][ProcFailedCompTask] Thread[140025572345600] recompile single op[Conv_84] failed[FUNC:ProcessAllFailedCompileTasks][FILE:tbe_op_store_adapter.cc][LINE:954] [SubGraphOpt][Compile][ProcFailedCompTask] Thread[140025572345600] recompile single op[Conv_77] failed[FUNC:ProcessAllFailedCompileTasks][FILE:tbe_op_store_adapter.cc][LINE:954] [SubGraphOpt][Compile][ProcFailedCompTask] Thread[140025572345600] recompile single op[Conv_53] failed[FUNC:ProcessAllFailedCompileTasks][FILE:tbe_op_store_adapter.cc][LINE:954] [SubGraphOpt][Compile][ProcFailedCompTask] Thread[140025572345600] recompile single op[Conv_59] failed[FUNC:ProcessAllFailedCompileTasks][FILE:tbe_op_store_adapter.cc][LINE:954] [SubGraphOpt][Compile][ProcFailedCompTask] Thread[140025572345600] recompile single op[Conv_66] failed[FUNC:ProcessAllFailedCompileTasks][FILE:tbe_op_store_adapter.cc][LINE:954] [SubGraphOpt][Compile][ProcFailedCompTask] Thread[140025572345600] recompile single op[Conv_73] failed[FUNC:ProcessAllFailedCompileTasks][FILE:tbe_op_store_adapter.cc][LINE:954] [SubGraphOpt][Compile][ProcFailedCompTask] Thread[140025572345600] recompile single op[Conv_56] failed[FUNC:ProcessAllFailedCompileTasks][FILE:tbe_op_store_adapter.cc][LINE:954] [SubGraphOpt][Compile][ProcFailedCompTask] Thread[140025572345600] recompile single op[Conv_23] failed[FUNC:ProcessAllFailedCompileTasks][FILE:tbe_op_store_adapter.cc][LINE:954] [SubGraphOpt][Compile][ProcFailedCompTask] Thread[140025572345600] recompile single op[Conv_63] failed[FUNC:ProcessAllFailedCompileTasks][FILE:tbe_op_store_adapter.cc][LINE:954] [SubGraphOpt][Compile][ProcFailedCompTask] Thread[140025572345600] recompile single op[Conv_9] failed[FUNC:ProcessAllFailedCompileTasks][FILE:tbe_op_store_adapter.cc][LINE:954] [SubGraphOpt][Compile][ProcFailedCompTask] Thread[140025572345600] recompile single op[Conv_70] failed[FUNC:ProcessAllFailedCompileTasks][FILE:tbe_op_store_adapter.cc][LINE:954] [SubGraphOpt][Compile][ProcFailedCompTask] Thread[140025572345600] recompile single op[Conv_114] failed[FUNC:ProcessAllFailedCompileTasks][FILE:tbe_op_store_adapter.cc][LINE:954] [SubGraphOpt][Compile][ProcFailedCompTask] Thread[140025572345600] recompile single op[Conv_81] failed[FUNC:ProcessAllFailedCompileTasks][FILE:tbe_op_store_adapter.cc][LINE:954] [SubGraphOpt][Compile][ParalCompOp] Thread[140025572345600] process fail task failed[FUNC:ParallelCompileOp][FILE:tbe_op_store_adapter.cc][LINE:1001] Call OptimizeFusedGraph failed, ret:-1, engine_name:AIcoreEngine, graph_name:partition0_rank1_new_sub_graph1[FUNC:OptimizeSubGraph][FILE:graph_optimize.cc][LINE:131] subgraph 0 optimize failed[FUNC:OptimizeSubGraphWithMultiThreads][FILE:graph_manager.cc][LINE:996] build graph failed, graph id:0, ret:-1[FUNC:BuildModelWithGraphId][FILE:ge_generator.cc][LINE:1615] GenerateOfflineModel execute failed. Compile operator failed, cause: 32Broken pipe File "/home/lsn/Ascend/ascend-toolkit/latest/python/site-packages/tbe/common/repository_manager/interface.py", line 42, in cann_kb_finalize RouteServer.finalize() File "/home/lsn/Ascend/ascend-toolkit/latest/python/site-packages/tbe/common/repository_manager/route.py", line 51, in wrapper return func(cls, *args, **kwargs) File "/home/lsn/Ascend/ascend-toolkit/latest/python/site-packages/tbe/common/repository_manager/route.py", line 212, in finalize cls.cann_kb_mgr.finalize() File "/home/lsn/Ascend/ascend-toolkit/latest/python/site-packages/tbe/common/repository_manager/cann_kb_manager/knowledge_bank_manager.py", line 41, in finalize self.__dynamic_mgr.close() File "/home/lsn/Ascend/ascend-toolkit/latest/python/site-packages/tbe/common/repository_manager/cann_kb_manager/base_manager.py", line 327, in close for name in copy.deepcopy(self.write_kb.keys()): File "<string>", line 2, in keys File "/usr/lib/python3.8/multiprocessing/managers.py", line 834, in _callmethod conn.send((self._id, methodname, args, kwds)) File "/usr/lib/python3.8/multiprocessing/connection.py", line 206, in send self._send_bytes(_ForkingPickler.dumps(obj)) File "/usr/lib/python3.8/multiprocessing/connection.py", line 411, in _send_bytes self._send(header + buf) File "/usr/lib/python3.8/multiprocessing/connection.py", line 368, in _send n = write(self._handle, buf)将虚拟机内存由8G增加到16G问题解决,可以正常转换模型