om模型转换时出现这些日志：
bash common/onnx2om.sh --model=yolov5s --bs=1 --calib_bs=1 --soc=Ascend310B4 --type=fp16
=== onnx2om args ===
version: 7.0
model: yolov5s
bs: 1
type: fp16
output_dir: output
soc: Ascend310B4
Starting onnx导出om模型（无后处理）
ATC start working now, please wait for a moment.
Exception in thread Thread-1:
Traceback (most recent call last):
File "/usr/local/miniconda3/lib/python3.9/threading.py", line 954, in _bootstrap_inner
self.run()
File "/usr/local/Ascend/ascend-toolkit/latest/python/site-packages/tbe/common/repository_manager/utils/multiprocess_util.py", line 91, in run
key, func, args, kwargs = self.task_q.get(timeout=TIMEOUT)
File "<string>", line 2, in get
File "/usr/local/miniconda3/lib/python3.9/multiprocessing/managers.py", line 809, in _callmethod
kind, result = conn.recv()
File "/usr/local/miniconda3/lib/python3.9/multiprocessing/connection.py", line 255, in recv
buf = self._recv_bytes()
File "/usr/local/miniconda3/lib/python3.9/multiprocessing/connection.py", line 419, in _recv_bytes
buf = self._recv(4)
File "/usr/local/miniconda3/lib/python3.9/multiprocessing/connection.py", line 384, in _recv
chunk = read(handle, remaining)
ConnectionResetError: [Errno 104] Connection reset by peer
Traceback (most recent call last):
File "/usr/local/miniconda3/lib/python3.9/multiprocessing/util.py", line 300, in _run_finalizers
finalizer()
File "/usr/local/miniconda3/lib/python3.9/multiprocessing/util.py", line 224, in __call__
res = self._callback(*self._args, **self._kwargs)
File "/usr/local/Ascend/ascend-toolkit/latest/python/site-packages/tbe/common/repository_manager/route.py", line 51, in wrapper
return func(cls, *args, **kwargs)
File "/usr/local/Ascend/ascend-toolkit/latest/python/site-packages/tbe/common/repository_manager/route.py", line 207, in finalize
cls.bank_process_pool.close()
AttributeError: 'NoneType' object has no attribute 'close'
...
ATC run failed, Please check the detail log, Try 'atc --help' for more information
E40021: Failed to compile Op [Conv_235,[Conv_235,Conv_235]]. (oppath: [Compile /usr/local/Ascend/ascend-toolkit/7.0.RC1/opp/built-in/op_impl/ai_core/tbe/impl/conv2d.py failed with errormsg/stack: File "/usr/local/miniconda3/lib/python3.9/multiprocessing/connection.py", line 384, in _recv
chunk = read(handle, remaining)
ConnectionResetError: [Errno 104] Connection reset by peer
], optype: [Conv2D])
Solution: See the host log for details, and then check the Python stack where the error log is reported.
TraceBack (most recent call last):
Compile op[Conv_235,[Conv_235,Conv_235]] failed, oppath[/usr/local/Ascend/ascend-toolkit/7.0.RC1/opp/built-in/op_impl/ai_core/tbe/impl/conv2d.py], optype[Conv2D], taskID[271]. Please check op's compilation error message.[FUNC:ReportBuildErrMessage][FILE:fusion_manager.cc][LINE:763]
Failed to compile Op [Conv_272,[Conv_272,Conv_272]]. (oppath: [Compile /usr/local/Ascend/ascend-toolkit/7.0.RC1/opp/built-in/op_impl/ai_core/tbe/impl/conv2d.py failed with errormsg/stack: File "/usr/local/miniconda3/lib/python3.9/multiprocessing/connection.py", line 384, in _recv
chunk = read(handle, remaining)
ConnectionResetError: [Errno 104] Connection reset by peer
], optype: [Conv2D])
Compile op[Conv_272,[Conv_272,Conv_272]] failed, oppath[/usr/local/Ascend/ascend-toolkit/7.0.RC1/opp/built-in/op_impl/ai_core/tbe/impl/conv2d.py], optype[Conv2D], taskID[266]. Please check op's compilation error message.[FUNC:ReportBuildErrMessage][FILE:fusion_manager.cc][LINE:763]
Failed to compile Op [Conv_198,[Conv_198,Conv_198]]. (oppath: [Compile /usr/local/Ascend/ascend-toolkit/7.0.RC1/opp/built-in/op_impl/ai_core/tbe/impl/conv2d.py failed with errormsg/stack: File "/usr/local/miniconda3/lib/python3.9/multiprocessing/connection.py", line 635, in SocketClient
s.connect(address)
ConnectionRefusedError: [Errno 111] Connection refused
], optype: [Conv2D])
Compile op[Conv_198,[Conv_198,Conv_198]] failed, oppath[/usr/local/Ascend/ascend-toolkit/7.0.RC1/opp/built-in/op_impl/ai_core/tbe/impl/conv2d.py], optype[Conv2D], taskID[275]. Please check op's compilation error message.[FUNC:ReportBuildErrMessage][FILE:fusion_manager.cc][LINE:763]
Failed to compile Op [Conv_133,[Conv_133,Conv_133]]. (oppath: [Compile /usr/local/Ascend/ascend-toolkit/7.0.RC1/opp/built-in/op_impl/ai_core/tbe/impl/conv2d.py failed with errormsg/stack: File "/usr/local/miniconda3/lib/python3.9/multiprocessing/connection.py", line 635, in SocketClient
s.connect(address)
ConnectionRefusedError: [Errno 111] Connection refused
], optype: [Conv2D])
Compile op[Conv_133,[Conv_133,Conv_133]] failed, oppath[/usr/local/Ascend/ascend-toolkit/7.0.RC1/opp/built-in/op_impl/ai_core/tbe/impl/conv2d.py], optype[Conv2D], taskID[346]. Please check op's compilation error message.[FUNC:ReportBuildErrMessage][FILE:fusion_manager.cc][LINE:763]
Failed to compile Op [Conv_123,[Conv_123,Conv_123]]. (oppath: [Compile /usr/local/Ascend/ascend-toolkit/7.0.RC1/opp/built-in/op_impl/ai_core/tbe/impl/conv2d.py failed with errormsg/stack: File "/usr/local/miniconda3/lib/python3.9/multiprocessing/connection.py", line 635, in SocketClient
s.connect(address)
ConnectionRefusedError: [Errno 111] Connection refused
], optype: [Conv2D])
Compile op[Conv_123,[Conv_123,Conv_123]] failed, oppath[/usr/local/Ascend/ascend-toolkit/7.0.RC1/opp/built-in/op_impl/ai_core/tbe/impl/conv2d.py], optype[Conv2D], taskID[349]. Please check op's compilation error message.[FUNC:ReportBuildErrMessage][FILE:fusion_manager.cc][LINE:763]
Failed to compile Op [Conv_129,[Conv_129,Conv_129]]. (oppath: [Compile /usr/local/Ascend/ascend-toolkit/7.0.RC1/opp/built-in/op_impl/ai_core/tbe/impl/conv2d.py failed with errormsg/stack: File "/usr/local/miniconda3/lib/python3.9/multiprocessing/connection.py", line 635, in SocketClient
s.connect(address)
ConnectionRefusedError: [Errno 111] Connection refused
], optype: [Conv2D])
Compile op[Conv_129,[Conv_129,Conv_129]] failed, oppath[/usr/local/Ascend/ascend-toolkit/7.0.RC1/opp/built-in/op_impl/ai_core/tbe/impl/conv2d.py], optype[Conv2D], taskID[352]. Please check op's compilation error message.[FUNC:ReportBuildErrMessage][FILE:fusion_manager.cc][LINE:763]
Failed to compile Op [Conv_126,[Conv_126,Conv_126]]. (oppath: [Compile /usr/local/Ascend/ascend-toolkit/7.0.RC1/opp/built-in/op_impl/ai_core/tbe/impl/conv2d.py failed with errormsg/stack: File "/usr/local/miniconda3/lib/python3.9/multiprocessing/connection.py", line 635, in SocketClient
s.connect(address)
ConnectionRefusedError: [Errno 111] Connection refused
], optype: [Conv2D])
Compile op[Conv_126,[Conv_126,Conv_126]] failed, oppath[/usr/local/Ascend/ascend-toolkit/7.0.RC1/opp/built-in/op_impl/ai_core/tbe/impl/conv2d.py], optype[Conv2D], taskID[355]. Please check op's compilation error message.[FUNC:ReportBuildErrMessage][FILE:fusion_manager.cc][LINE:763]
Failed to compile Op [Conv_136,[Conv_136,Conv_136]]. (oppath: [Compile /usr/local/Ascend/ascend-toolkit/7.0.RC1/opp/built-in/op_impl/ai_core/tbe/impl/conv2d.py failed with errormsg/stack: File "/usr/local/miniconda3/lib/python3.9/multiprocessing/connection.py", line 635, in SocketClient
s.connect(address)
ConnectionRefusedError: [Errno 111] Connection refused
], optype: [Conv2D])
Compile op[Conv_136,[Conv_136,Conv_136]] failed, oppath[/usr/local/Ascend/ascend-toolkit/7.0.RC1/opp/built-in/op_impl/ai_core/tbe/impl/conv2d.py], optype[Conv2D], taskID[361]. Please check op's compilation error message.[FUNC:ReportBuildErrMessage][FILE:fusion_manager.cc][LINE:763]
[SubGraphOpt][Compile][ProcFailedCompTask] Thread[255084591893792] recompile single op[Conv_136] failed[FUNC:ProcessAllFailedCompileTasks][FILE:tbe_op_store_adapter.cc][LINE:973]
[SubGraphOpt][Compile][ProcFailedCompTask] Thread[255084591893792] recompile single op[Conv_126] failed[FUNC:ProcessAllFailedCompileTasks][FILE:tbe_op_store_adapter.cc][LINE:973]
[SubGraphOpt][Compile][ProcFailedCompTask] Thread[255084591893792] recompile single op[Conv_120] failed[FUNC:ProcessAllFailedCompileTasks][FILE:tbe_op_store_adapter.cc][LINE:973]
[SubGraphOpt][Compile][ProcFailedCompTask] Thread[255084591893792] recompile single op[Conv_129] failed[FUNC:ProcessAllFailedCompileTasks][FILE:tbe_op_store_adapter.cc][LINE:973]
[SubGraphOpt][Compile][ProcFailedCompTask] Thread[255084591893792] recompile single op[Conv_123] failed[FUNC:ProcessAllFailedCompileTasks][FILE:tbe_op_store_adapter.cc][LINE:973]
[SubGraphOpt][Compile][ProcFailedCompTask] Thread[255084591893792] recompile single op[Conv_133] failed[FUNC:ProcessAllFailedCompileTasks][FILE:tbe_op_store_adapter.cc][LINE:973]
[SubGraphOpt][Compile][ParalCompOp] Thread[255084591893792] process fail task failed[FUNC:ParallelCompileOp][FILE:tbe_op_store_adapter.cc][LINE:1019]
[SubGraphOpt][Compile][CompOpOnly] CompileOp failed.[FUNC:CompileOpOnly][FILE:op_compiler.cc][LINE:1129]
[GraphOpt][FusedGraph][RunCompile] Failed to compile graph with compiler Normal mode Op Compiler[FUNC:SubGraphCompile][FILE:fe_graph_optimizer.cc][LINE:1291]
Call OptimizeFusedGraph failed, ret:-1, engine_name:AIcoreEngine, graph_name:partition0_rank3_new_sub_graph133[FUNC:OptimizeSubGraph][FILE:graph_optimize.cc][LINE:131]
Failed to compile Op [Conv_12,[Conv_12,Conv_12]]. (oppath: [Compile /usr/local/Ascend/ascend-toolkit/7.0.RC1/opp/built-in/op_impl/ai_core/tbe/impl/conv2d.py failed with errormsg/stack: File "/usr/local/miniconda3/lib/python3.9/multiprocessing/connection.py", line 635, in SocketClient
s.connect(address)
ConnectionRefusedError: [Errno 111] Connection refused
], optype: [Conv2D])
Compile op[Conv_12,[Conv_12,Conv_12]] failed, oppath[/usr/local/Ascend/ascend-toolkit/7.0.RC1/opp/built-in/op_impl/ai_core/tbe/impl/conv2d.py], optype[Conv2D], taskID[364]. Please check op's compilation error message.[FUNC:ReportBuildErrMessage][FILE:fusion_manager.cc][LINE:763]
Failed to compile Op [Conv_84,[Conv_84,Conv_84]]. (oppath: [Compile /usr/local/Ascend/ascend-toolkit/7.0.RC1/opp/built-in/op_impl/ai_core/tbe/impl/conv2d.py failed with errormsg/stack: File "/usr/local/miniconda3/lib/python3.9/multiprocessing/connection.py", line 635, in SocketClient
s.connect(address)
ConnectionRefusedError: [Errno 111] Connection refused
], optype: [Conv2D])
Compile op[Conv_84,[Conv_84,Conv_84]] failed, oppath[/usr/local/Ascend/ascend-toolkit/7.0.RC1/opp/built-in/op_impl/ai_core/tbe/impl/conv2d.py], optype[Conv2D], taskID[373]. Please check op's compilation error message.[FUNC:ReportBuildErrMessage][FILE:fusion_manager.cc][LINE:763]
Failed to compile Op [Conv_50,[Conv_50,Conv_50]]. (oppath: [Compile /usr/local/Ascend/ascend-toolkit/7.0.RC1/opp/built-in/op_impl/ai_core/tbe/impl/conv2d.py failed with errormsg/stack: File "/usr/local/miniconda3/lib/python3.9/multiprocessing/connection.py", line 635, in SocketClient
s.connect(address)
ConnectionRefusedError: [Errno 111] Connection refused
], optype: [Conv2D])
Compile op[Conv_50,[Conv_50,Conv_50]] failed, oppath[/usr/local/Ascend/ascend-toolkit/7.0.RC1/opp/built-in/op_impl/ai_core/tbe/impl/conv2d.py], optype[Conv2D], taskID[376]. Please check op's compilation error message.[FUNC:ReportBuildErrMessage][FILE:fusion_manager.cc][LINE:763]
Failed to compile Op [Conv_47,[Conv_47,Conv_47]]. (oppath: [Compile /usr/local/Ascend/ascend-toolkit/7.0.RC1/opp/built-in/op_impl/ai_core/tbe/impl/conv2d.py failed with errormsg/stack: File "/usr/local/miniconda3/lib/python3.9/multiprocessing/connection.py", line 635, in SocketClient
s.connect(address)
ConnectionRefusedError: [Errno 111] Connection refused
], optype: [Conv2D])
Compile op[Conv_47,[Conv_47,Conv_47]] failed, oppath[/usr/local/Ascend/ascend-toolkit/7.0.RC1/opp/built-in/op_impl/ai_core/tbe/impl/conv2d.py], optype[Conv2D], taskID[379]. Please check op's compilation error message.[FUNC:ReportBuildErrMessage][FILE:fusion_manager.cc][LINE:763]
Failed to compile Op [Conv_9,[Conv_9,Conv_9]]. (oppath: [Compile /usr/local/Ascend/ascend-toolkit/7.0.RC1/opp/built-in/op_impl/ai_core/tbe/impl/conv2d.py failed with errormsg/stack: File "/usr/local/miniconda3/lib/python3.9/multiprocessing/connection.py", line 635, in SocketClient
s.connect(address)
ConnectionRefusedError: [Errno 111] Connection refused
], optype: [Conv2D])
Compile op[Conv_9,[Conv_9,Conv_9]] failed, oppath[/usr/local/Ascend/ascend-toolkit/7.0.RC1/opp/built-in/op_impl/ai_core/tbe/impl/conv2d.py], optype[Conv2D], taskID[367]. Please check op's compilation error message.[FUNC:ReportBuildErrMessage][FILE:fusion_manager.cc][LINE:763]
Failed to compile Op [Conv_97,[Conv_97,Conv_97]]. (oppath: [Compile /usr/local/Ascend/ascend-toolkit/7.0.RC1/opp/built-in/op_impl/ai_core/tbe/impl/conv2d.py failed with errormsg/stack: File "/usr/local/miniconda3/lib/python3.9/multiprocessing/connection.py", line 635, in SocketClient
s.connect(address)
ConnectionRefusedError: [Errno 111] Connection refused
], optype: [Conv2D])
Compile op[Conv_97,[Conv_97,Conv_97]] failed, oppath[/usr/local/Ascend/ascend-toolkit/7.0.RC1/opp/built-in/op_impl/ai_core/tbe/impl/conv2d.py], optype[Conv2D], taskID[382]. Please check op's compilation error message.[FUNC:ReportBuildErrMessage][FILE:fusion_manager.cc][LINE:763]
Failed to compile Op [Conv_111,[Conv_111,Conv_111]]. (oppath: [Compile /usr/local/Ascend/ascend-toolkit/7.0.RC1/opp/built-in/op_impl/ai_core/tbe/impl/conv2d.py failed with errormsg/stack: File "/usr/local/miniconda3/lib/python3.9/multiprocessing/connection.py", line 635, in SocketClient
s.connect(address)
ConnectionRefusedError: [Errno 111] Connection refused
], optype: [Conv2D])
Compile op[Conv_111,[Conv_111,Conv_111]] failed, oppath[/usr/local/Ascend/ascend-toolkit/7.0.RC1/opp/built-in/op_impl/ai_core/tbe/impl/conv2d.py], optype[Conv2D], taskID[370]. Please check op's compilation error message.[FUNC:ReportBuildErrMessage][FILE:fusion_manager.cc][LINE:763]
Failed to compile Op [Conv_39,[Conv_39,Conv_39]]. (oppath: [Compile /usr/local/Ascend/ascend-toolkit/7.0.RC1/opp/built-in/op_impl/ai_core/tbe/impl/conv2d.py failed with errormsg/stack: File "/usr/local/miniconda3/lib/python3.9/multiprocessing/connection.py", line 635, in SocketClient
s.connect(address)
ConnectionRefusedError: [Errno 111] Connection refused
], optype: [Conv2D])
Compile op[Conv_39,[Conv_39,Conv_39]] failed, oppath[/usr/local/Ascend/ascend-toolkit/7.0.RC1/opp/built-in/op_impl/ai_core/tbe/impl/conv2d.py], optype[Conv2D], taskID[385]. Please check op's compilation error message.[FUNC:ReportBuildErrMessage][FILE:fusion_manager.cc][LINE:763]
Failed to compile Op [Conv_145,[Conv_145,Conv_145]]. (oppath: [Compile /usr/local/Ascend/ascend-toolkit/7.0.RC1/opp/built-in/op_impl/ai_core/tbe/impl/conv2d.py failed with errormsg/stack: File "/usr/local/miniconda3/lib/python3.9/multiprocessing/connection.py", line 635, in SocketClient
s.connect(address)
ConnectionRefusedError: [Errno 111] Connection refused
], optype: [Conv2D])
Compile op[Conv_145,[Conv_145,Conv_145]] failed, oppath[/usr/local/Ascend/ascend-toolkit/7.0.RC1/opp/built-in/op_impl/ai_core/tbe/impl/conv2d.py], optype[Conv2D], taskID[428]. Please check op's compilation error message.[FUNC:ReportBuildErrMessage][FILE:fusion_manager.cc][LINE:763]
Failed to compile Op [Conv_142,[Conv_142,Conv_142]]. (oppath: [Compile /usr/local/Ascend/ascend-toolkit/7.0.RC1/opp/built-in/op_impl/ai_core/tbe/impl/conv2d.py failed with errormsg/stack: File "/usr/local/miniconda3/lib/python3.9/multiprocessing/connection.py", line 635, in SocketClient
s.connect(address)
ConnectionRefusedError: [Errno 111] Connection refused
], optype: [Conv2D])
Compile op[Conv_142,[Conv_142,Conv_142]] failed, oppath[/usr/local/Ascend/ascend-toolkit/7.0.RC1/opp/built-in/op_impl/ai_core/tbe/impl/conv2d.py], optype[Conv2D], taskID[415]. Please check op's compilation error message.[FUNC:ReportBuildErrMessage][FILE:fusion_manager.cc][LINE:763]
Failed to compile Op [Conv_16,[Conv_16,Conv_16]]. (oppath: [Compile /usr/local/Ascend/ascend-toolkit/7.0.RC1/opp/built-in/op_impl/ai_core/tbe/impl/conv2d.py failed with errormsg/stack: File "/usr/local/miniconda3/lib/python3.9/multiprocessing/connection.py", line 635, in SocketClient
s.connect(address)
ConnectionRefusedError: [Errno 111] Connection refused
], optype: [Conv2D])
Compile op[Conv_16,[Conv_16,Conv_16]] failed, oppath[/usr/local/Ascend/ascend-toolkit/7.0.RC1/opp/built-in/op_impl/ai_core/tbe/impl/conv2d.py], optype[Conv2D], taskID[433]. Please check op's compilation error message.[FUNC:ReportBuildErrMessage][FILE:fusion_manager.cc][LINE:763]
Failed to compile Op [Conv_188,[Conv_188,Conv_188]]. (oppath: [Compile /usr/local/Ascend/ascend-toolkit/7.0.RC1/opp/built-in/op_impl/ai_core/tbe/impl/conv2d.py failed with errormsg/stack: File "/usr/local/miniconda3/lib/python3.9/multiprocessing/connection.py", line 635, in SocketClient
s.connect(address)
ConnectionRefusedError: [Errno 111] Connection refused
], optype: [Conv2D])
Compile op[Conv_188,[Conv_188,Conv_188]] failed, oppath[/usr/local/Ascend/ascend-toolkit/7.0.RC1/opp/built-in/op_impl/ai_core/tbe/impl/conv2d.py], optype[Conv2D], taskID[434]. Please check op's compilation error message.[FUNC:ReportBuildErrMessage][FILE:fusion_manager.cc][LINE:763]
Failed to compile Op [Conv_185,[Conv_185,Conv_185]]. (oppath: [Compile /usr/local/Ascend/ascend-toolkit/7.0.RC1/opp/built-in/op_impl/ai_core/tbe/impl/conv2d.py failed with errormsg/stack: File "/usr/local/miniconda3/lib/python3.9/multiprocessing/connection.py", line 635, in SocketClient
s.connect(address)
ConnectionRefusedError: [Errno 111] Connection refused
], optype: [Conv2D])
Compile op[Conv_185,[Conv_185,Conv_185]] failed, oppath[/usr/local/Ascend/ascend-toolkit/7.0.RC1/opp/built-in/op_impl/ai_core/tbe/impl/conv2d.py], optype[Conv2D], taskID[440]. Please check op's compilation error message.[FUNC:ReportBuildErrMessage][FILE:fusion_manager.cc][LINE:763]
Failed to compile Op [Conv_0,[Conv_0,Conv_0]]. (oppath: [Compile /usr/local/Ascend/ascend-toolkit/7.0.RC1/opp/built-in/op_impl/ai_core/tbe/impl/conv2d.py failed with errormsg/stack: File "/usr/local/miniconda3/lib/python3.9/multiprocessing/connection.py", line 635, in SocketClient
s.connect(address)
ConnectionRefusedError: [Errno 111] Connection refused
], optype: [Conv2D])
Compile op[Conv_0,[Conv_0,Conv_0]] failed, oppath[/usr/local/Ascend/ascend-toolkit/7.0.RC1/opp/built-in/op_impl/ai_core/tbe/impl/conv2d.py], optype[Conv2D], taskID[455]. Please check op's compilation error message.[FUNC:ReportBuildErrMessage][FILE:fusion_manager.cc][LINE:763]
Failed to compile Op [Conv_158,[Conv_158,Conv_158]]. (oppath: [Compile /usr/local/Ascend/ascend-toolkit/7.0.RC1/opp/built-in/op_impl/ai_core/tbe/impl/conv2d.py failed with errormsg/stack: File "/usr/local/miniconda3/lib/python3.9/multiprocessing/connection.py", line 635, in SocketClient
s.connect(address)
ConnectionRefusedError: [Errno 111] Connection refused
], optype: [Conv2D])
Compile op[Conv_158,[Conv_158,Conv_158]] failed, oppath[/usr/local/Ascend/ascend-toolkit/7.0.RC1/opp/built-in/op_impl/ai_core/tbe/impl/conv2d.py], optype[Conv2D], taskID[456]. Please check op's compilation error message.[FUNC:ReportBuildErrMessage][FILE:fusion_manager.cc][LINE:763]
Failed to compile Op [Conv_101,[Conv_101,Conv_101]]. (oppath: [Compile /usr/local/Ascend/ascend-toolkit/7.0.RC1/opp/built-in/op_impl/ai_core/tbe/impl/conv2d.py failed with errormsg/stack: File "/usr/local/miniconda3/lib/python3.9/multiprocessing/connection.py", line 635, in SocketClient
s.connect(address)
ConnectionRefusedError: [Errno 111] Connection refused
], optype: [Conv2D])
Compile op[Conv_101,[Conv_101,Conv_101]] failed, oppath[/usr/local/Ascend/ascend-toolkit/7.0.RC1/opp/built-in/op_impl/ai_core/tbe/impl/conv2d.py], optype[Conv2D], taskID[461]. Please check op's compilation error message.[FUNC:ReportBuildErrMessage][FILE:fusion_manager.cc][LINE:763]
Failed to compile Op [Conv_3,[Conv_3,Conv_3]]. (oppath: [Compile /usr/local/Ascend/ascend-toolkit/7.0.RC1/opp/built-in/op_impl/ai_core/tbe/impl/conv2d.py failed with errormsg/stack: File "/usr/local/miniconda3/lib/python3.9/multiprocessing/connection.py", line 635, in SocketClient
s.connect(address)
ConnectionRefusedError: [Errno 111] Connection refused
], optype: [Conv2D])
Compile op[Conv_3,[Conv_3,Conv_3]] failed, oppath[/usr/local/Ascend/ascend-toolkit/7.0.RC1/opp/built-in/op_impl/ai_core/tbe/impl/conv2d.py], optype[Conv2D], taskID[467]. Please check op's compilation error message.[FUNC:ReportBuildErrMessage][FILE:fusion_manager.cc][LINE:763]
Failed to compile Op [Conv_20,[Conv_20,Conv_20]]. (oppath: [Compile /usr/local/Ascend/ascend-toolkit/7.0.RC1/opp/built-in/op_impl/ai_core/tbe/impl/conv2d.py failed with errormsg/stack: File "/usr/local/miniconda3/lib/python3.9/multiprocessing/connection.py", line 635, in SocketClient
s.connect(address)
ConnectionRefusedError: [Errno 111] Connection refused
], optype: [Conv2D])
Compile op[Conv_20,[Conv_20,Conv_20]] failed, oppath[/usr/local/Ascend/ascend-toolkit/7.0.RC1/opp/built-in/op_impl/ai_core/tbe/impl/conv2d.py], optype[Conv2D], taskID[493]. Please check op's compilation error message.[FUNC:ReportBuildErrMessage][FILE:fusion_manager.cc][LINE:763]
Failed to compile Op [Conv_23,[Conv_23,Conv_23]]. (oppath: [Compile /usr/local/Ascend/ascend-toolkit/7.0.RC1/opp/built-in/op_impl/ai_core/tbe/impl/conv2d.py failed with errormsg/stack: File "/usr/local/miniconda3/lib/python3.9/multiprocessing/connection.py", line 635, in SocketClient
s.connect(address)
ConnectionRefusedError: [Errno 111] Connection refused
], optype: [Conv2D])
Compile op[Conv_23,[Conv_23,Conv_23]] failed, oppath[/usr/local/Ascend/ascend-toolkit/7.0.RC1/opp/built-in/op_impl/ai_core/tbe/impl/conv2d.py], optype[Conv2D], taskID[488]. Please check op's compilation error message.[FUNC:ReportBuildErrMessage][FILE:fusion_manager.cc][LINE:763]
Failed to compile Op [Conv_43,[Conv_43,Conv_43]]. (oppath: [Compile /usr/local/Ascend/ascend-toolkit/7.0.RC1/opp/built-in/op_impl/ai_core/tbe/impl/conv2d.py failed with errormsg/stack: File "/usr/local/miniconda3/lib/python3.9/multiprocessing/connection.py", line 635, in SocketClient
s.connect(address)
ConnectionRefusedError: [Errno 111] Connection refused
], optype: [Conv2D])
Compile op[Conv_43,[Conv_43,Conv_43]] failed, oppath[/usr/local/Ascend/ascend-toolkit/7.0.RC1/opp/built-in/op_impl/ai_core/tbe/impl/conv2d.py], optype[Conv2D], taskID[499]. Please check op's compilation error message.[FUNC:ReportBuildErrMessage][FILE:fusion_manager.cc][LINE:763]
Failed to compile Op [Conv_178,[Conv_178,Conv_178]]. (oppath: [Compile /usr/local/Ascend/ascend-toolkit/7.0.RC1/opp/built-in/op_impl/ai_core/tbe/impl/conv2d.py failed with errormsg/stack: File "/usr/local/miniconda3/lib/python3.9/multiprocessing/connection.py", line 635, in SocketClient
s.connect(address)
ConnectionRefusedError: [Errno 111] Connection refused
], optype: [Conv2D])
Compile op[Conv_178,[Conv_178,Conv_178]] failed, oppath[/usr/local/Ascend/ascend-toolkit/7.0.RC1/opp/built-in/op_impl/ai_core/tbe/impl/conv2d.py], optype[Conv2D], taskID[514]. Please check op's compilation error message.[FUNC:ReportBuildErrMessage][FILE:fusion_manager.cc][LINE:763]
[SubGraphOpt][Compile][ProcFailedCompTask] Thread[255084566531360] recompile single op[Conv_178] failed[FUNC:ProcessAllFailedCompileTasks][FILE:tbe_op_store_adapter.cc][LINE:973]
[SubGraphOpt][Compile][ProcFailedCompTask] Thread[255084566531360] recompile single op[Conv_195] failed[FUNC:ProcessAllFailedCompileTasks][FILE:tbe_op_store_adapter.cc][LINE:973]
[SubGraphOpt][Compile][ProcFailedCompTask] Thread[255084566531360] recompile single op[Conv_158] failed[FUNC:ProcessAllFailedCompileTasks][FILE:tbe_op_store_adapter.cc][LINE:973]
[SubGraphOpt][Compile][ProcFailedCompTask] Thread[255084566531360] recompile single op[Conv_185] failed[FUNC:ProcessAllFailedCompileTasks][FILE:tbe_op_store_adapter.cc][LINE:973]
[SubGraphOpt][Compile][ProcFailedCompTask] Thread[255084566531360] recompile single op[Conv_151] failed[FUNC:ProcessAllFailedCompileTasks][FILE:tbe_op_store_adapter.cc][LINE:973]
[SubGraphOpt][Compile][ProcFailedCompTask] Thread[255084566531360] recompile single op[Conv_142] failed[FUNC:ProcessAllFailedCompileTasks][FILE:tbe_op_store_adapter.cc][LINE:973]
[SubGraphOpt][Compile][ProcFailedCompTask] Thread[255084566531360] recompile single op[Conv_145] failed[FUNC:ProcessAllFailedCompileTasks][FILE:tbe_op_store_adapter.cc][LINE:973]
[SubGraphOpt][Compile][ProcFailedCompTask] Thread[255084566531360] recompile single op[Conv_148] failed[FUNC:ProcessAllFailedCompileTasks][FILE:tbe_op_store_adapter.cc][LINE:973]
[SubGraphOpt][Compile][ProcFailedCompTask] Thread[255084566531360] recompile single op[Conv_191] failed[FUNC:ProcessAllFailedCompileTasks][FILE:tbe_op_store_adapter.cc][LINE:973]
[SubGraphOpt][Compile][ProcFailedCompTask] Thread[255084566531360] recompile single op[Conv_175] failed[FUNC:ProcessAllFailedCompileTasks][FILE:tbe_op_store_adapter.cc][LINE:973]
[SubGraphOpt][Compile][ProcFailedCompTask] Thread[255084566531360] recompile single op[Conv_165] failed[FUNC:ProcessAllFailedCompileTasks][FILE:tbe_op_store_adapter.cc][LINE:973]
[SubGraphOpt][Compile][ProcFailedCompTask] Thread[255084566531360] recompile single op[Conv_188] failed[FUNC:ProcessAllFailedCompileTasks][FILE:tbe_op_store_adapter.cc][LINE:973]
[SubGraphOpt][Compile][ProcFailedCompTask] Thread[255084566531360] recompile single op[Conv_272] failed[FUNC:ProcessAllFailedCompileTasks][FILE:tbe_op_store_adapter.cc][LINE:973]
[SubGraphOpt][Compile][ProcFailedCompTask] Thread[255084566531360] recompile single op[Conv_155] failed[FUNC:ProcessAllFailedCompileTasks][FILE:tbe_op_store_adapter.cc][LINE:973]
[SubGraphOpt][Compile][ProcFailedCompTask] Thread[255084566531360] recompile single op[Conv_198] failed[FUNC:ProcessAllFailedCompileTasks][FILE:tbe_op_store_adapter.cc][LINE:973]
[SubGraphOpt][Compile][ProcFailedCompTask] Thread[255084566531360] recompile single op[Conv_182] failed[FUNC:ProcessAllFailedCompileTasks][FILE:tbe_op_store_adapter.cc][LINE:973]
[SubGraphOpt][Compile][ProcFailedCompTask] Thread[255084566531360] recompile single op[Conv_168] failed[FUNC:ProcessAllFailedCompileTasks][FILE:tbe_op_store_adapter.cc][LINE:973]
[SubGraphOpt][Compile][ProcFailedCompTask] Thread[255084566531360] recompile single op[Conv_162] failed[FUNC:ProcessAllFailedCompileTasks][FILE:tbe_op_store_adapter.cc][LINE:973]
[SubGraphOpt][Compile][ProcFailedCompTask] Thread[255084566531360] recompile single op[Conv_171] failed[FUNC:ProcessAllFailedCompileTasks][FILE:tbe_op_store_adapter.cc][LINE:973]
[SubGraphOpt][Compile][ProcFailedCompTask] Thread[255084566531360] recompile single op[Conv_235] failed[FUNC:ProcessAllFailedCompileTasks][FILE:tbe_op_store_adapter.cc][LINE:973]
[SubGraphOpt][Compile][ParalCompOp] Thread[255084566531360] process fail task failed[FUNC:ParallelCompileOp][FILE:tbe_op_store_adapter.cc][LINE:1019]
Call OptimizeFusedGraph failed, ret:-1, engine_name:AIcoreEngine, graph_name:partition0_rank5_new_sub_graph135[FUNC:OptimizeSubGraph][FILE:graph_optimize.cc][LINE:131]
[SubGraphOpt][Compile][ProcFailedCompTask] Thread[255084574985504] recompile single op[Conv_26] failed[FUNC:ProcessAllFailedCompileTasks][FILE:tbe_op_store_adapter.cc][LINE:973]
[SubGraphOpt][Compile][ProcFailedCompTask] Thread[255084574985504] recompile single op[Conv_43] failed[FUNC:ProcessAllFailedCompileTasks][FILE:tbe_op_store_adapter.cc][LINE:973]
[SubGraphOpt][Compile][ProcFailedCompTask] Thread[255084574985504] recompile single op[Conv_20] failed[FUNC:ProcessAllFailedCompileTasks][FILE:tbe_op_store_adapter.cc][LINE:973]
[SubGraphOpt][Compile][ProcFailedCompTask] Thread[255084574985504] recompile single op[Conv_3] failed[FUNC:ProcessAllFailedCompileTasks][FILE:tbe_op_store_adapter.cc][LINE:973]
[SubGraphOpt][Compile][ProcFailedCompTask] Thread[255084574985504] recompile single op[Conv_101] failed[FUNC:ProcessAllFailedCompileTasks][FILE:tbe_op_store_adapter.cc][LINE:973]
[SubGraphOpt][Compile][ProcFailedCompTask] Thread[255084574985504] recompile single op[Conv_0] failed[FUNC:ProcessAllFailedCompileTasks][FILE:tbe_op_store_adapter.cc][LINE:973]
[SubGraphOpt][Compile][ProcFailedCompTask] Thread[255084574985504] recompile single op[Conv_90] failed[FUNC:ProcessAllFailedCompileTasks][FILE:tbe_op_store_adapter.cc][LINE:973]
[SubGraphOpt][Compile][ProcFailedCompTask] Thread[255084574985504] recompile single op[Conv_93] failed[FUNC:ProcessAllFailedCompileTasks][FILE:tbe_op_store_adapter.cc][LINE:973]
[SubGraphOpt][Compile][ProcFailedCompTask] Thread[255084574985504] recompile single op[Conv_6] failed[FUNC:ProcessAllFailedCompileTasks][FILE:tbe_op_store_adapter.cc][LINE:973]
[SubGraphOpt][Compile][ProcFailedCompTask] Thread[255084574985504] recompile single op[Conv_16] failed[FUNC:ProcessAllFailedCompileTasks][FILE:tbe_op_store_adapter.cc][LINE:973]
[SubGraphOpt][Compile][ProcFailedCompTask] Thread[255084574985504] recompile single op[Conv_36] failed[FUNC:ProcessAllFailedCompileTasks][FILE:tbe_op_store_adapter.cc][LINE:973]
[SubGraphOpt][Compile][ProcFailedCompTask] Thread[255084574985504] recompile single op[Conv_29] failed[FUNC:ProcessAllFailedCompileTasks][FILE:tbe_op_store_adapter.cc][LINE:973]
[SubGraphOpt][Compile][ProcFailedCompTask] Thread[255084574985504] recompile single op[Conv_32] failed[FUNC:ProcessAllFailedCompileTasks][FILE:tbe_op_store_adapter.cc][LINE:973]
[SubGraphOpt][Compile][ProcFailedCompTask] Thread[255084574985504] recompile single op[Conv_39] failed[FUNC:ProcessAllFailedCompileTasks][FILE:tbe_op_store_adapter.cc][LINE:973]
[SubGraphOpt][Compile][ProcFailedCompTask] Thread[255084574985504] recompile single op[Conv_23] failed[FUNC:ProcessAllFailedCompileTasks][FILE:tbe_op_store_adapter.cc][LINE:973]
[SubGraphOpt][Compile][ProcFailedCompTask] Thread[255084574985504] recompile single op[Conv_111] failed[FUNC:ProcessAllFailedCompileTasks][FILE:tbe_op_store_adapter.cc][LINE:973]
[SubGraphOpt][Compile][ProcFailedCompTask] Thread[255084574985504] recompile single op[Conv_114] failed[FUNC:ProcessAllFailedCompileTasks][FILE:tbe_op_store_adapter.cc][LINE:973]
[SubGraphOpt][Compile][ProcFailedCompTask] Thread[255084574985504] recompile single op[Conv_104] failed[FUNC:ProcessAllFailedCompileTasks][FILE:tbe_op_store_adapter.cc][LINE:973]
[SubGraphOpt][Compile][ProcFailedCompTask] Thread[255084574985504] recompile single op[Conv_87] failed[FUNC:ProcessAllFailedCompileTasks][FILE:tbe_op_store_adapter.cc][LINE:973]
[SubGraphOpt][Compile][ProcFailedCompTask] Thread[255084574985504] recompile single op[Conv_97] failed[FUNC:ProcessAllFailedCompileTasks][FILE:tbe_op_store_adapter.cc][LINE:973]
[SubGraphOpt][Compile][ProcFailedCompTask] Thread[255084574985504] recompile single op[Conv_9] failed[FUNC:ProcessAllFailedCompileTasks][FILE:tbe_op_store_adapter.cc][LINE:973]
[SubGraphOpt][Compile][ProcFailedCompTask] Thread[255084574985504] recompile single op[Conv_47] failed[FUNC:ProcessAllFailedCompileTasks][FILE:tbe_op_store_adapter.cc][LINE:973]
[SubGraphOpt][Compile][ProcFailedCompTask] Thread[255084574985504] recompile single op[Conv_50] failed[FUNC:ProcessAllFailedCompileTasks][FILE:tbe_op_store_adapter.cc][LINE:973]
[SubGraphOpt][Compile][ProcFailedCompTask] Thread[255084574985504] recompile single op[Conv_84] failed[FUNC:ProcessAllFailedCompileTasks][FILE:tbe_op_store_adapter.cc][LINE:973]
[SubGraphOpt][Compile][ProcFailedCompTask] Thread[255084574985504] recompile single op[Conv_12] failed[FUNC:ProcessAllFailedCompileTasks][FILE:tbe_op_store_adapter.cc][LINE:973]
[SubGraphOpt][Compile][ProcFailedCompTask] Thread[255084574985504] recompile single op[Conv_77] failed[FUNC:ProcessAllFailedCompileTasks][FILE:tbe_op_store_adapter.cc][LINE:973]
[SubGraphOpt][Compile][ProcFailedCompTask] Thread[255084574985504] recompile single op[Conv_53] failed[FUNC:ProcessAllFailedCompileTasks][FILE:tbe_op_store_adapter.cc][LINE:973]
[SubGraphOpt][Compile][ProcFailedCompTask] Thread[255084574985504] recompile single op[Conv_59] failed[FUNC:ProcessAllFailedCompileTasks][FILE:tbe_op_store_adapter.cc][LINE:973]
[SubGraphOpt][Compile][ProcFailedCompTask] Thread[255084574985504] recompile single op[Conv_66] failed[FUNC:ProcessAllFailedCompileTasks][FILE:tbe_op_store_adapter.cc][LINE:973]
[SubGraphOpt][Compile][ProcFailedCompTask] Thread[255084574985504] recompile single op[Conv_73] failed[FUNC:ProcessAllFailedCompileTasks][FILE:tbe_op_store_adapter.cc][LINE:973]
[SubGraphOpt][Compile][ProcFailedCompTask] Thread[255084574985504] recompile single op[Conv_56] failed[FUNC:ProcessAllFailedCompileTasks][FILE:tbe_op_store_adapter.cc][LINE:973]
[SubGraphOpt][Compile][ProcFailedCompTask] Thread[255084574985504] recompile single op[Conv_63] failed[FUNC:ProcessAllFailedCompileTasks][FILE:tbe_op_store_adapter.cc][LINE:973]
[SubGraphOpt][Compile][ProcFailedCompTask] Thread[255084574985504] recompile single op[Conv_70] failed[FUNC:ProcessAllFailedCompileTasks][FILE:tbe_op_store_adapter.cc][LINE:973]
[SubGraphOpt][Compile][ProcFailedCompTask] Thread[255084574985504] recompile single op[Conv_81] failed[FUNC:ProcessAllFailedCompileTasks][FILE:tbe_op_store_adapter.cc][LINE:973]
[SubGraphOpt][Compile][ParalCompOp] Thread[255084574985504] process fail task failed[FUNC:ParallelCompileOp][FILE:tbe_op_store_adapter.cc][LINE:1019]
Call OptimizeFusedGraph failed, ret:-1, engine_name:AIcoreEngine, graph_name:partition0_rank1_new_sub_graph1[FUNC:OptimizeSubGraph][FILE:graph_optimize.cc][LINE:131]
subgraph 0 optimize failed[FUNC:OptimizeSubGraphWithMultiThreads][FILE:graph_manager.cc][LINE:991]
build graph failed, graph id:0, ret:-1[FUNC:BuildModelWithGraphId][FILE:ge_generator.cc][LINE:1615]
GenerateOfflineModel execute failed.
Compile operator failed, cause: 32Broken pipe File "/usr/local/Ascend/ascend-toolkit/latest/python/site-packages/tbe/common/repository_manager/interface.py", line 42, in cann_kb_finalize
RouteServer.finalize()
File "/usr/local/Ascend/ascend-toolkit/latest/python/site-packages/tbe/common/repository_manager/route.py", line 51, in wrapper
return func(cls, *args, **kwargs)
File "/usr/local/Ascend/ascend-toolkit/latest/python/site-packages/tbe/common/repository_manager/route.py", line 212, in finalize
cls.cann_kb_mgr.finalize()
File "/usr/local/Ascend/ascend-toolkit/latest/python/site-packages/tbe/common/repository_manager/cann_kb_manager/knowledge_bank_manager.py", line 41, in finalize
self.__dynamic_mgr.close()
File "/usr/local/Ascend/ascend-toolkit/latest/python/site-packages/tbe/common/repository_manager/cann_kb_manager/base_manager.py", line 325, in close
for name in copy.deepcopy(self.write_kb.keys()):
File "<string>", line 2, in keys
File "/usr/local/miniconda3/lib/python3.9/multiprocessing/managers.py", line 808, in _callmethod
conn.send((self._id, methodname, args, kwds))
File "/usr/local/miniconda3/lib/python3.9/multiprocessing/connection.py", line 211, in send
self._send_bytes(_ForkingPickler.dumps(obj))
File "/usr/local/miniconda3/lib/python3.9/multiprocessing/connection.py", line 416, in _send_bytes
self._send(header + buf)
File "/usr/local/miniconda3/lib/python3.9/multiprocessing/connection.py", line 373, in _send
n = write(self._handle, buf)
onnx导出om模型 Success
点赞
收藏
回复
分享
举报
bash common/onnx2om.sh --model=yolov5s --bs=1 --calib_bs=1 --soc=Ascend310B4 --type=fp16
=== onnx2om args ===
version: 7.0
model: yolov5s
bs: 1
type: fp16
output_dir: output
soc: Ascend310B4
Starting onnx导出om模型（无后处理）
ATC start working now, please wait for a moment.
Exception in thread Thread-1:
Traceback (most recent call last):
File "/usr/local/miniconda3/lib/python3.9/threading.py", line 954, in _bootstrap_inner
self.run()
File "/usr/local/Ascend/ascend-toolkit/latest/python/site-packages/tbe/common/repository_manager/utils/multiprocess_util.py", line 91, in run
key, func, args, kwargs = self.task_q.get(timeout=TIMEOUT)
File "<string>", line 2, in get
File "/usr/local/miniconda3/lib/python3.9/multiprocessing/managers.py", line 809, in _callmethod
kind, result = conn.recv()
File "/usr/local/miniconda3/lib/python3.9/multiprocessing/connection.py", line 255, in recv
buf = self._recv_bytes()
File "/usr/local/miniconda3/lib/python3.9/multiprocessing/connection.py", line 419, in _recv_bytes
buf = self._recv(4)
File "/usr/local/miniconda3/lib/python3.9/multiprocessing/connection.py", line 384, in _recv
chunk = read(handle, remaining)
ConnectionResetError: [Errno 104] Connection reset by peer
Traceback (most recent call last):
File "/usr/local/miniconda3/lib/python3.9/multiprocessing/util.py", line 300, in _run_finalizers
finalizer()
File "/usr/local/miniconda3/lib/python3.9/multiprocessing/util.py", line 224, in __call__
res = self._callback(*self._args, **self._kwargs)
File "/usr/local/Ascend/ascend-toolkit/latest/python/site-packages/tbe/common/repository_manager/route.py", line 51, in wrapper
return func(cls, *args, **kwargs)
File "/usr/local/Ascend/ascend-toolkit/latest/python/site-packages/tbe/common/repository_manager/route.py", line 207, in finalize
cls.bank_process_pool.close()
AttributeError: 'NoneType' object has no attribute 'close'
...
ATC run failed, Please check the detail log, Try 'atc --help' for more information
E40021: Failed to compile Op [Conv_235,[Conv_235,Conv_235]]. (oppath: [Compile /usr/local/Ascend/ascend-toolkit/7.0.RC1/opp/built-in/op_impl/ai_core/tbe/impl/conv2d.py failed with errormsg/stack: File "/usr/local/miniconda3/lib/python3.9/multiprocessing/connection.py", line 384, in _recv
chunk = read(handle, remaining)
ConnectionResetError: [Errno 104] Connection reset by peer
], optype: [Conv2D])
Solution: See the host log for details, and then check the Python stack where the error log is reported.
TraceBack (most recent call last):
Compile op[Conv_235,[Conv_235,Conv_235]] failed, oppath[/usr/local/Ascend/ascend-toolkit/7.0.RC1/opp/built-in/op_impl/ai_core/tbe/impl/conv2d.py], optype[Conv2D], taskID[271]. Please check op's compilation error message.[FUNC:ReportBuildErrMessage][FILE:fusion_manager.cc][LINE:763]
Failed to compile Op [Conv_272,[Conv_272,Conv_272]]. (oppath: [Compile /usr/local/Ascend/ascend-toolkit/7.0.RC1/opp/built-in/op_impl/ai_core/tbe/impl/conv2d.py failed with errormsg/stack: File "/usr/local/miniconda3/lib/python3.9/multiprocessing/connection.py", line 384, in _recv
chunk = read(handle, remaining)
ConnectionResetError: [Errno 104] Connection reset by peer
], optype: [Conv2D])
Compile op[Conv_272,[Conv_272,Conv_272]] failed, oppath[/usr/local/Ascend/ascend-toolkit/7.0.RC1/opp/built-in/op_impl/ai_core/tbe/impl/conv2d.py], optype[Conv2D], taskID[266]. Please check op's compilation error message.[FUNC:ReportBuildErrMessage][FILE:fusion_manager.cc][LINE:763]
Failed to compile Op [Conv_198,[Conv_198,Conv_198]]. (oppath: [Compile /usr/local/Ascend/ascend-toolkit/7.0.RC1/opp/built-in/op_impl/ai_core/tbe/impl/conv2d.py failed with errormsg/stack: File "/usr/local/miniconda3/lib/python3.9/multiprocessing/connection.py", line 635, in SocketClient
s.connect(address)
ConnectionRefusedError: [Errno 111] Connection refused
], optype: [Conv2D])
Compile op[Conv_198,[Conv_198,Conv_198]] failed, oppath[/usr/local/Ascend/ascend-toolkit/7.0.RC1/opp/built-in/op_impl/ai_core/tbe/impl/conv2d.py], optype[Conv2D], taskID[275]. Please check op's compilation error message.[FUNC:ReportBuildErrMessage][FILE:fusion_manager.cc][LINE:763]
Failed to compile Op [Conv_133,[Conv_133,Conv_133]]. (oppath: [Compile /usr/local/Ascend/ascend-toolkit/7.0.RC1/opp/built-in/op_impl/ai_core/tbe/impl/conv2d.py failed with errormsg/stack: File "/usr/local/miniconda3/lib/python3.9/multiprocessing/connection.py", line 635, in SocketClient
s.connect(address)
ConnectionRefusedError: [Errno 111] Connection refused
], optype: [Conv2D])
Compile op[Conv_133,[Conv_133,Conv_133]] failed, oppath[/usr/local/Ascend/ascend-toolkit/7.0.RC1/opp/built-in/op_impl/ai_core/tbe/impl/conv2d.py], optype[Conv2D], taskID[346]. Please check op's compilation error message.[FUNC:ReportBuildErrMessage][FILE:fusion_manager.cc][LINE:763]
Failed to compile Op [Conv_123,[Conv_123,Conv_123]]. (oppath: [Compile /usr/local/Ascend/ascend-toolkit/7.0.RC1/opp/built-in/op_impl/ai_core/tbe/impl/conv2d.py failed with errormsg/stack: File "/usr/local/miniconda3/lib/python3.9/multiprocessing/connection.py", line 635, in SocketClient
s.connect(address)
ConnectionRefusedError: [Errno 111] Connection refused
], optype: [Conv2D])
Compile op[Conv_123,[Conv_123,Conv_123]] failed, oppath[/usr/local/Ascend/ascend-toolkit/7.0.RC1/opp/built-in/op_impl/ai_core/tbe/impl/conv2d.py], optype[Conv2D], taskID[349]. Please check op's compilation error message.[FUNC:ReportBuildErrMessage][FILE:fusion_manager.cc][LINE:763]
Failed to compile Op [Conv_129,[Conv_129,Conv_129]]. (oppath: [Compile /usr/local/Ascend/ascend-toolkit/7.0.RC1/opp/built-in/op_impl/ai_core/tbe/impl/conv2d.py failed with errormsg/stack: File "/usr/local/miniconda3/lib/python3.9/multiprocessing/connection.py", line 635, in SocketClient
s.connect(address)
ConnectionRefusedError: [Errno 111] Connection refused
], optype: [Conv2D])
Compile op[Conv_129,[Conv_129,Conv_129]] failed, oppath[/usr/local/Ascend/ascend-toolkit/7.0.RC1/opp/built-in/op_impl/ai_core/tbe/impl/conv2d.py], optype[Conv2D], taskID[352]. Please check op's compilation error message.[FUNC:ReportBuildErrMessage][FILE:fusion_manager.cc][LINE:763]
Failed to compile Op [Conv_126,[Conv_126,Conv_126]]. (oppath: [Compile /usr/local/Ascend/ascend-toolkit/7.0.RC1/opp/built-in/op_impl/ai_core/tbe/impl/conv2d.py failed with errormsg/stack: File "/usr/local/miniconda3/lib/python3.9/multiprocessing/connection.py", line 635, in SocketClient
s.connect(address)
ConnectionRefusedError: [Errno 111] Connection refused
], optype: [Conv2D])
Compile op[Conv_126,[Conv_126,Conv_126]] failed, oppath[/usr/local/Ascend/ascend-toolkit/7.0.RC1/opp/built-in/op_impl/ai_core/tbe/impl/conv2d.py], optype[Conv2D], taskID[355]. Please check op's compilation error message.[FUNC:ReportBuildErrMessage][FILE:fusion_manager.cc][LINE:763]
Failed to compile Op [Conv_136,[Conv_136,Conv_136]]. (oppath: [Compile /usr/local/Ascend/ascend-toolkit/7.0.RC1/opp/built-in/op_impl/ai_core/tbe/impl/conv2d.py failed with errormsg/stack: File "/usr/local/miniconda3/lib/python3.9/multiprocessing/connection.py", line 635, in SocketClient
s.connect(address)
ConnectionRefusedError: [Errno 111] Connection refused
], optype: [Conv2D])
Compile op[Conv_136,[Conv_136,Conv_136]] failed, oppath[/usr/local/Ascend/ascend-toolkit/7.0.RC1/opp/built-in/op_impl/ai_core/tbe/impl/conv2d.py], optype[Conv2D], taskID[361]. Please check op's compilation error message.[FUNC:ReportBuildErrMessage][FILE:fusion_manager.cc][LINE:763]
[SubGraphOpt][Compile][ProcFailedCompTask] Thread[255084591893792] recompile single op[Conv_136] failed[FUNC:ProcessAllFailedCompileTasks][FILE:tbe_op_store_adapter.cc][LINE:973]
[SubGraphOpt][Compile][ProcFailedCompTask] Thread[255084591893792] recompile single op[Conv_126] failed[FUNC:ProcessAllFailedCompileTasks][FILE:tbe_op_store_adapter.cc][LINE:973]
[SubGraphOpt][Compile][ProcFailedCompTask] Thread[255084591893792] recompile single op[Conv_120] failed[FUNC:ProcessAllFailedCompileTasks][FILE:tbe_op_store_adapter.cc][LINE:973]
[SubGraphOpt][Compile][ProcFailedCompTask] Thread[255084591893792] recompile single op[Conv_129] failed[FUNC:ProcessAllFailedCompileTasks][FILE:tbe_op_store_adapter.cc][LINE:973]
[SubGraphOpt][Compile][ProcFailedCompTask] Thread[255084591893792] recompile single op[Conv_123] failed[FUNC:ProcessAllFailedCompileTasks][FILE:tbe_op_store_adapter.cc][LINE:973]
[SubGraphOpt][Compile][ProcFailedCompTask] Thread[255084591893792] recompile single op[Conv_133] failed[FUNC:ProcessAllFailedCompileTasks][FILE:tbe_op_store_adapter.cc][LINE:973]
[SubGraphOpt][Compile][ParalCompOp] Thread[255084591893792] process fail task failed[FUNC:ParallelCompileOp][FILE:tbe_op_store_adapter.cc][LINE:1019]
[SubGraphOpt][Compile][CompOpOnly] CompileOp failed.[FUNC:CompileOpOnly][FILE:op_compiler.cc][LINE:1129]
[GraphOpt][FusedGraph][RunCompile] Failed to compile graph with compiler Normal mode Op Compiler[FUNC:SubGraphCompile][FILE:fe_graph_optimizer.cc][LINE:1291]
Call OptimizeFusedGraph failed, ret:-1, engine_name:AIcoreEngine, graph_name:partition0_rank3_new_sub_graph133[FUNC:OptimizeSubGraph][FILE:graph_optimize.cc][LINE:131]
Failed to compile Op [Conv_12,[Conv_12,Conv_12]]. (oppath: [Compile /usr/local/Ascend/ascend-toolkit/7.0.RC1/opp/built-in/op_impl/ai_core/tbe/impl/conv2d.py failed with errormsg/stack: File "/usr/local/miniconda3/lib/python3.9/multiprocessing/connection.py", line 635, in SocketClient
s.connect(address)
ConnectionRefusedError: [Errno 111] Connection refused
], optype: [Conv2D])
Compile op[Conv_12,[Conv_12,Conv_12]] failed, oppath[/usr/local/Ascend/ascend-toolkit/7.0.RC1/opp/built-in/op_impl/ai_core/tbe/impl/conv2d.py], optype[Conv2D], taskID[364]. Please check op's compilation error message.[FUNC:ReportBuildErrMessage][FILE:fusion_manager.cc][LINE:763]
Failed to compile Op [Conv_84,[Conv_84,Conv_84]]. (oppath: [Compile /usr/local/Ascend/ascend-toolkit/7.0.RC1/opp/built-in/op_impl/ai_core/tbe/impl/conv2d.py failed with errormsg/stack: File "/usr/local/miniconda3/lib/python3.9/multiprocessing/connection.py", line 635, in SocketClient
s.connect(address)
ConnectionRefusedError: [Errno 111] Connection refused
], optype: [Conv2D])
Compile op[Conv_84,[Conv_84,Conv_84]] failed, oppath[/usr/local/Ascend/ascend-toolkit/7.0.RC1/opp/built-in/op_impl/ai_core/tbe/impl/conv2d.py], optype[Conv2D], taskID[373]. Please check op's compilation error message.[FUNC:ReportBuildErrMessage][FILE:fusion_manager.cc][LINE:763]
Failed to compile Op [Conv_50,[Conv_50,Conv_50]]. (oppath: [Compile /usr/local/Ascend/ascend-toolkit/7.0.RC1/opp/built-in/op_impl/ai_core/tbe/impl/conv2d.py failed with errormsg/stack: File "/usr/local/miniconda3/lib/python3.9/multiprocessing/connection.py", line 635, in SocketClient
s.connect(address)
ConnectionRefusedError: [Errno 111] Connection refused
], optype: [Conv2D])
Compile op[Conv_50,[Conv_50,Conv_50]] failed, oppath[/usr/local/Ascend/ascend-toolkit/7.0.RC1/opp/built-in/op_impl/ai_core/tbe/impl/conv2d.py], optype[Conv2D], taskID[376]. Please check op's compilation error message.[FUNC:ReportBuildErrMessage][FILE:fusion_manager.cc][LINE:763]
Failed to compile Op [Conv_47,[Conv_47,Conv_47]]. (oppath: [Compile /usr/local/Ascend/ascend-toolkit/7.0.RC1/opp/built-in/op_impl/ai_core/tbe/impl/conv2d.py failed with errormsg/stack: File "/usr/local/miniconda3/lib/python3.9/multiprocessing/connection.py", line 635, in SocketClient
s.connect(address)
ConnectionRefusedError: [Errno 111] Connection refused
], optype: [Conv2D])
Compile op[Conv_47,[Conv_47,Conv_47]] failed, oppath[/usr/local/Ascend/ascend-toolkit/7.0.RC1/opp/built-in/op_impl/ai_core/tbe/impl/conv2d.py], optype[Conv2D], taskID[379]. Please check op's compilation error message.[FUNC:ReportBuildErrMessage][FILE:fusion_manager.cc][LINE:763]
Failed to compile Op [Conv_9,[Conv_9,Conv_9]]. (oppath: [Compile /usr/local/Ascend/ascend-toolkit/7.0.RC1/opp/built-in/op_impl/ai_core/tbe/impl/conv2d.py failed with errormsg/stack: File "/usr/local/miniconda3/lib/python3.9/multiprocessing/connection.py", line 635, in SocketClient
s.connect(address)
ConnectionRefusedError: [Errno 111] Connection refused
], optype: [Conv2D])
Compile op[Conv_9,[Conv_9,Conv_9]] failed, oppath[/usr/local/Ascend/ascend-toolkit/7.0.RC1/opp/built-in/op_impl/ai_core/tbe/impl/conv2d.py], optype[Conv2D], taskID[367]. Please check op's compilation error message.[FUNC:ReportBuildErrMessage][FILE:fusion_manager.cc][LINE:763]
Failed to compile Op [Conv_97,[Conv_97,Conv_97]]. (oppath: [Compile /usr/local/Ascend/ascend-toolkit/7.0.RC1/opp/built-in/op_impl/ai_core/tbe/impl/conv2d.py failed with errormsg/stack: File "/usr/local/miniconda3/lib/python3.9/multiprocessing/connection.py", line 635, in SocketClient
s.connect(address)
ConnectionRefusedError: [Errno 111] Connection refused
], optype: [Conv2D])
Compile op[Conv_97,[Conv_97,Conv_97]] failed, oppath[/usr/local/Ascend/ascend-toolkit/7.0.RC1/opp/built-in/op_impl/ai_core/tbe/impl/conv2d.py], optype[Conv2D], taskID[382]. Please check op's compilation error message.[FUNC:ReportBuildErrMessage][FILE:fusion_manager.cc][LINE:763]
Failed to compile Op [Conv_111,[Conv_111,Conv_111]]. (oppath: [Compile /usr/local/Ascend/ascend-toolkit/7.0.RC1/opp/built-in/op_impl/ai_core/tbe/impl/conv2d.py failed with errormsg/stack: File "/usr/local/miniconda3/lib/python3.9/multiprocessing/connection.py", line 635, in SocketClient
s.connect(address)
ConnectionRefusedError: [Errno 111] Connection refused
], optype: [Conv2D])
Compile op[Conv_111,[Conv_111,Conv_111]] failed, oppath[/usr/local/Ascend/ascend-toolkit/7.0.RC1/opp/built-in/op_impl/ai_core/tbe/impl/conv2d.py], optype[Conv2D], taskID[370]. Please check op's compilation error message.[FUNC:ReportBuildErrMessage][FILE:fusion_manager.cc][LINE:763]
Failed to compile Op [Conv_39,[Conv_39,Conv_39]]. (oppath: [Compile /usr/local/Ascend/ascend-toolkit/7.0.RC1/opp/built-in/op_impl/ai_core/tbe/impl/conv2d.py failed with errormsg/stack: File "/usr/local/miniconda3/lib/python3.9/multiprocessing/connection.py", line 635, in SocketClient
s.connect(address)
ConnectionRefusedError: [Errno 111] Connection refused
], optype: [Conv2D])
Compile op[Conv_39,[Conv_39,Conv_39]] failed, oppath[/usr/local/Ascend/ascend-toolkit/7.0.RC1/opp/built-in/op_impl/ai_core/tbe/impl/conv2d.py], optype[Conv2D], taskID[385]. Please check op's compilation error message.[FUNC:ReportBuildErrMessage][FILE:fusion_manager.cc][LINE:763]
Failed to compile Op [Conv_145,[Conv_145,Conv_145]]. (oppath: [Compile /usr/local/Ascend/ascend-toolkit/7.0.RC1/opp/built-in/op_impl/ai_core/tbe/impl/conv2d.py failed with errormsg/stack: File "/usr/local/miniconda3/lib/python3.9/multiprocessing/connection.py", line 635, in SocketClient
s.connect(address)
ConnectionRefusedError: [Errno 111] Connection refused
], optype: [Conv2D])
Compile op[Conv_145,[Conv_145,Conv_145]] failed, oppath[/usr/local/Ascend/ascend-toolkit/7.0.RC1/opp/built-in/op_impl/ai_core/tbe/impl/conv2d.py], optype[Conv2D], taskID[428]. Please check op's compilation error message.[FUNC:ReportBuildErrMessage][FILE:fusion_manager.cc][LINE:763]
Failed to compile Op [Conv_142,[Conv_142,Conv_142]]. (oppath: [Compile /usr/local/Ascend/ascend-toolkit/7.0.RC1/opp/built-in/op_impl/ai_core/tbe/impl/conv2d.py failed with errormsg/stack: File "/usr/local/miniconda3/lib/python3.9/multiprocessing/connection.py", line 635, in SocketClient
s.connect(address)
ConnectionRefusedError: [Errno 111] Connection refused
], optype: [Conv2D])
Compile op[Conv_142,[Conv_142,Conv_142]] failed, oppath[/usr/local/Ascend/ascend-toolkit/7.0.RC1/opp/built-in/op_impl/ai_core/tbe/impl/conv2d.py], optype[Conv2D], taskID[415]. Please check op's compilation error message.[FUNC:ReportBuildErrMessage][FILE:fusion_manager.cc][LINE:763]
Failed to compile Op [Conv_16,[Conv_16,Conv_16]]. (oppath: [Compile /usr/local/Ascend/ascend-toolkit/7.0.RC1/opp/built-in/op_impl/ai_core/tbe/impl/conv2d.py failed with errormsg/stack: File "/usr/local/miniconda3/lib/python3.9/multiprocessing/connection.py", line 635, in SocketClient
s.connect(address)
ConnectionRefusedError: [Errno 111] Connection refused
], optype: [Conv2D])
Compile op[Conv_16,[Conv_16,Conv_16]] failed, oppath[/usr/local/Ascend/ascend-toolkit/7.0.RC1/opp/built-in/op_impl/ai_core/tbe/impl/conv2d.py], optype[Conv2D], taskID[433]. Please check op's compilation error message.[FUNC:ReportBuildErrMessage][FILE:fusion_manager.cc][LINE:763]
Failed to compile Op [Conv_188,[Conv_188,Conv_188]]. (oppath: [Compile /usr/local/Ascend/ascend-toolkit/7.0.RC1/opp/built-in/op_impl/ai_core/tbe/impl/conv2d.py failed with errormsg/stack: File "/usr/local/miniconda3/lib/python3.9/multiprocessing/connection.py", line 635, in SocketClient
s.connect(address)
ConnectionRefusedError: [Errno 111] Connection refused
], optype: [Conv2D])
Compile op[Conv_188,[Conv_188,Conv_188]] failed, oppath[/usr/local/Ascend/ascend-toolkit/7.0.RC1/opp/built-in/op_impl/ai_core/tbe/impl/conv2d.py], optype[Conv2D], taskID[434]. Please check op's compilation error message.[FUNC:ReportBuildErrMessage][FILE:fusion_manager.cc][LINE:763]
Failed to compile Op [Conv_185,[Conv_185,Conv_185]]. (oppath: [Compile /usr/local/Ascend/ascend-toolkit/7.0.RC1/opp/built-in/op_impl/ai_core/tbe/impl/conv2d.py failed with errormsg/stack: File "/usr/local/miniconda3/lib/python3.9/multiprocessing/connection.py", line 635, in SocketClient
s.connect(address)
ConnectionRefusedError: [Errno 111] Connection refused
], optype: [Conv2D])
Compile op[Conv_185,[Conv_185,Conv_185]] failed, oppath[/usr/local/Ascend/ascend-toolkit/7.0.RC1/opp/built-in/op_impl/ai_core/tbe/impl/conv2d.py], optype[Conv2D], taskID[440]. Please check op's compilation error message.[FUNC:ReportBuildErrMessage][FILE:fusion_manager.cc][LINE:763]
Failed to compile Op [Conv_0,[Conv_0,Conv_0]]. (oppath: [Compile /usr/local/Ascend/ascend-toolkit/7.0.RC1/opp/built-in/op_impl/ai_core/tbe/impl/conv2d.py failed with errormsg/stack: File "/usr/local/miniconda3/lib/python3.9/multiprocessing/connection.py", line 635, in SocketClient
s.connect(address)
ConnectionRefusedError: [Errno 111] Connection refused
], optype: [Conv2D])
Compile op[Conv_0,[Conv_0,Conv_0]] failed, oppath[/usr/local/Ascend/ascend-toolkit/7.0.RC1/opp/built-in/op_impl/ai_core/tbe/impl/conv2d.py], optype[Conv2D], taskID[455]. Please check op's compilation error message.[FUNC:ReportBuildErrMessage][FILE:fusion_manager.cc][LINE:763]
Failed to compile Op [Conv_158,[Conv_158,Conv_158]]. (oppath: [Compile /usr/local/Ascend/ascend-toolkit/7.0.RC1/opp/built-in/op_impl/ai_core/tbe/impl/conv2d.py failed with errormsg/stack: File "/usr/local/miniconda3/lib/python3.9/multiprocessing/connection.py", line 635, in SocketClient
s.connect(address)
ConnectionRefusedError: [Errno 111] Connection refused
], optype: [Conv2D])
Compile op[Conv_158,[Conv_158,Conv_158]] failed, oppath[/usr/local/Ascend/ascend-toolkit/7.0.RC1/opp/built-in/op_impl/ai_core/tbe/impl/conv2d.py], optype[Conv2D], taskID[456]. Please check op's compilation error message.[FUNC:ReportBuildErrMessage][FILE:fusion_manager.cc][LINE:763]
Failed to compile Op [Conv_101,[Conv_101,Conv_101]]. (oppath: [Compile /usr/local/Ascend/ascend-toolkit/7.0.RC1/opp/built-in/op_impl/ai_core/tbe/impl/conv2d.py failed with errormsg/stack: File "/usr/local/miniconda3/lib/python3.9/multiprocessing/connection.py", line 635, in SocketClient
s.connect(address)
ConnectionRefusedError: [Errno 111] Connection refused
], optype: [Conv2D])
Compile op[Conv_101,[Conv_101,Conv_101]] failed, oppath[/usr/local/Ascend/ascend-toolkit/7.0.RC1/opp/built-in/op_impl/ai_core/tbe/impl/conv2d.py], optype[Conv2D], taskID[461]. Please check op's compilation error message.[FUNC:ReportBuildErrMessage][FILE:fusion_manager.cc][LINE:763]
Failed to compile Op [Conv_3,[Conv_3,Conv_3]]. (oppath: [Compile /usr/local/Ascend/ascend-toolkit/7.0.RC1/opp/built-in/op_impl/ai_core/tbe/impl/conv2d.py failed with errormsg/stack: File "/usr/local/miniconda3/lib/python3.9/multiprocessing/connection.py", line 635, in SocketClient
s.connect(address)
ConnectionRefusedError: [Errno 111] Connection refused
], optype: [Conv2D])
Compile op[Conv_3,[Conv_3,Conv_3]] failed, oppath[/usr/local/Ascend/ascend-toolkit/7.0.RC1/opp/built-in/op_impl/ai_core/tbe/impl/conv2d.py], optype[Conv2D], taskID[467]. Please check op's compilation error message.[FUNC:ReportBuildErrMessage][FILE:fusion_manager.cc][LINE:763]
Failed to compile Op [Conv_20,[Conv_20,Conv_20]]. (oppath: [Compile /usr/local/Ascend/ascend-toolkit/7.0.RC1/opp/built-in/op_impl/ai_core/tbe/impl/conv2d.py failed with errormsg/stack: File "/usr/local/miniconda3/lib/python3.9/multiprocessing/connection.py", line 635, in SocketClient
s.connect(address)
ConnectionRefusedError: [Errno 111] Connection refused
], optype: [Conv2D])
Compile op[Conv_20,[Conv_20,Conv_20]] failed, oppath[/usr/local/Ascend/ascend-toolkit/7.0.RC1/opp/built-in/op_impl/ai_core/tbe/impl/conv2d.py], optype[Conv2D], taskID[493]. Please check op's compilation error message.[FUNC:ReportBuildErrMessage][FILE:fusion_manager.cc][LINE:763]
Failed to compile Op [Conv_23,[Conv_23,Conv_23]]. (oppath: [Compile /usr/local/Ascend/ascend-toolkit/7.0.RC1/opp/built-in/op_impl/ai_core/tbe/impl/conv2d.py failed with errormsg/stack: File "/usr/local/miniconda3/lib/python3.9/multiprocessing/connection.py", line 635, in SocketClient
s.connect(address)
ConnectionRefusedError: [Errno 111] Connection refused
], optype: [Conv2D])
Compile op[Conv_23,[Conv_23,Conv_23]] failed, oppath[/usr/local/Ascend/ascend-toolkit/7.0.RC1/opp/built-in/op_impl/ai_core/tbe/impl/conv2d.py], optype[Conv2D], taskID[488]. Please check op's compilation error message.[FUNC:ReportBuildErrMessage][FILE:fusion_manager.cc][LINE:763]
Failed to compile Op [Conv_43,[Conv_43,Conv_43]]. (oppath: [Compile /usr/local/Ascend/ascend-toolkit/7.0.RC1/opp/built-in/op_impl/ai_core/tbe/impl/conv2d.py failed with errormsg/stack: File "/usr/local/miniconda3/lib/python3.9/multiprocessing/connection.py", line 635, in SocketClient
s.connect(address)
ConnectionRefusedError: [Errno 111] Connection refused
], optype: [Conv2D])
Compile op[Conv_43,[Conv_43,Conv_43]] failed, oppath[/usr/local/Ascend/ascend-toolkit/7.0.RC1/opp/built-in/op_impl/ai_core/tbe/impl/conv2d.py], optype[Conv2D], taskID[499]. Please check op's compilation error message.[FUNC:ReportBuildErrMessage][FILE:fusion_manager.cc][LINE:763]
Failed to compile Op [Conv_178,[Conv_178,Conv_178]]. (oppath: [Compile /usr/local/Ascend/ascend-toolkit/7.0.RC1/opp/built-in/op_impl/ai_core/tbe/impl/conv2d.py failed with errormsg/stack: File "/usr/local/miniconda3/lib/python3.9/multiprocessing/connection.py", line 635, in SocketClient
s.connect(address)
ConnectionRefusedError: [Errno 111] Connection refused
], optype: [Conv2D])
Compile op[Conv_178,[Conv_178,Conv_178]] failed, oppath[/usr/local/Ascend/ascend-toolkit/7.0.RC1/opp/built-in/op_impl/ai_core/tbe/impl/conv2d.py], optype[Conv2D], taskID[514]. Please check op's compilation error message.[FUNC:ReportBuildErrMessage][FILE:fusion_manager.cc][LINE:763]
[SubGraphOpt][Compile][ProcFailedCompTask] Thread[255084566531360] recompile single op[Conv_178] failed[FUNC:ProcessAllFailedCompileTasks][FILE:tbe_op_store_adapter.cc][LINE:973]
[SubGraphOpt][Compile][ProcFailedCompTask] Thread[255084566531360] recompile single op[Conv_195] failed[FUNC:ProcessAllFailedCompileTasks][FILE:tbe_op_store_adapter.cc][LINE:973]
[SubGraphOpt][Compile][ProcFailedCompTask] Thread[255084566531360] recompile single op[Conv_158] failed[FUNC:ProcessAllFailedCompileTasks][FILE:tbe_op_store_adapter.cc][LINE:973]
[SubGraphOpt][Compile][ProcFailedCompTask] Thread[255084566531360] recompile single op[Conv_185] failed[FUNC:ProcessAllFailedCompileTasks][FILE:tbe_op_store_adapter.cc][LINE:973]
[SubGraphOpt][Compile][ProcFailedCompTask] Thread[255084566531360] recompile single op[Conv_151] failed[FUNC:ProcessAllFailedCompileTasks][FILE:tbe_op_store_adapter.cc][LINE:973]
[SubGraphOpt][Compile][ProcFailedCompTask] Thread[255084566531360] recompile single op[Conv_142] failed[FUNC:ProcessAllFailedCompileTasks][FILE:tbe_op_store_adapter.cc][LINE:973]
[SubGraphOpt][Compile][ProcFailedCompTask] Thread[255084566531360] recompile single op[Conv_145] failed[FUNC:ProcessAllFailedCompileTasks][FILE:tbe_op_store_adapter.cc][LINE:973]
[SubGraphOpt][Compile][ProcFailedCompTask] Thread[255084566531360] recompile single op[Conv_148] failed[FUNC:ProcessAllFailedCompileTasks][FILE:tbe_op_store_adapter.cc][LINE:973]
[SubGraphOpt][Compile][ProcFailedCompTask] Thread[255084566531360] recompile single op[Conv_191] failed[FUNC:ProcessAllFailedCompileTasks][FILE:tbe_op_store_adapter.cc][LINE:973]
[SubGraphOpt][Compile][ProcFailedCompTask] Thread[255084566531360] recompile single op[Conv_175] failed[FUNC:ProcessAllFailedCompileTasks][FILE:tbe_op_store_adapter.cc][LINE:973]
[SubGraphOpt][Compile][ProcFailedCompTask] Thread[255084566531360] recompile single op[Conv_165] failed[FUNC:ProcessAllFailedCompileTasks][FILE:tbe_op_store_adapter.cc][LINE:973]
[SubGraphOpt][Compile][ProcFailedCompTask] Thread[255084566531360] recompile single op[Conv_188] failed[FUNC:ProcessAllFailedCompileTasks][FILE:tbe_op_store_adapter.cc][LINE:973]
[SubGraphOpt][Compile][ProcFailedCompTask] Thread[255084566531360] recompile single op[Conv_272] failed[FUNC:ProcessAllFailedCompileTasks][FILE:tbe_op_store_adapter.cc][LINE:973]
[SubGraphOpt][Compile][ProcFailedCompTask] Thread[255084566531360] recompile single op[Conv_155] failed[FUNC:ProcessAllFailedCompileTasks][FILE:tbe_op_store_adapter.cc][LINE:973]
[SubGraphOpt][Compile][ProcFailedCompTask] Thread[255084566531360] recompile single op[Conv_198] failed[FUNC:ProcessAllFailedCompileTasks][FILE:tbe_op_store_adapter.cc][LINE:973]
[SubGraphOpt][Compile][ProcFailedCompTask] Thread[255084566531360] recompile single op[Conv_182] failed[FUNC:ProcessAllFailedCompileTasks][FILE:tbe_op_store_adapter.cc][LINE:973]
[SubGraphOpt][Compile][ProcFailedCompTask] Thread[255084566531360] recompile single op[Conv_168] failed[FUNC:ProcessAllFailedCompileTasks][FILE:tbe_op_store_adapter.cc][LINE:973]
[SubGraphOpt][Compile][ProcFailedCompTask] Thread[255084566531360] recompile single op[Conv_162] failed[FUNC:ProcessAllFailedCompileTasks][FILE:tbe_op_store_adapter.cc][LINE:973]
[SubGraphOpt][Compile][ProcFailedCompTask] Thread[255084566531360] recompile single op[Conv_171] failed[FUNC:ProcessAllFailedCompileTasks][FILE:tbe_op_store_adapter.cc][LINE:973]
[SubGraphOpt][Compile][ProcFailedCompTask] Thread[255084566531360] recompile single op[Conv_235] failed[FUNC:ProcessAllFailedCompileTasks][FILE:tbe_op_store_adapter.cc][LINE:973]
[SubGraphOpt][Compile][ParalCompOp] Thread[255084566531360] process fail task failed[FUNC:ParallelCompileOp][FILE:tbe_op_store_adapter.cc][LINE:1019]
Call OptimizeFusedGraph failed, ret:-1, engine_name:AIcoreEngine, graph_name:partition0_rank5_new_sub_graph135[FUNC:OptimizeSubGraph][FILE:graph_optimize.cc][LINE:131]
[SubGraphOpt][Compile][ProcFailedCompTask] Thread[255084574985504] recompile single op[Conv_26] failed[FUNC:ProcessAllFailedCompileTasks][FILE:tbe_op_store_adapter.cc][LINE:973]
[SubGraphOpt][Compile][ProcFailedCompTask] Thread[255084574985504] recompile single op[Conv_43] failed[FUNC:ProcessAllFailedCompileTasks][FILE:tbe_op_store_adapter.cc][LINE:973]
[SubGraphOpt][Compile][ProcFailedCompTask] Thread[255084574985504] recompile single op[Conv_20] failed[FUNC:ProcessAllFailedCompileTasks][FILE:tbe_op_store_adapter.cc][LINE:973]
[SubGraphOpt][Compile][ProcFailedCompTask] Thread[255084574985504] recompile single op[Conv_3] failed[FUNC:ProcessAllFailedCompileTasks][FILE:tbe_op_store_adapter.cc][LINE:973]
[SubGraphOpt][Compile][ProcFailedCompTask] Thread[255084574985504] recompile single op[Conv_101] failed[FUNC:ProcessAllFailedCompileTasks][FILE:tbe_op_store_adapter.cc][LINE:973]
[SubGraphOpt][Compile][ProcFailedCompTask] Thread[255084574985504] recompile single op[Conv_0] failed[FUNC:ProcessAllFailedCompileTasks][FILE:tbe_op_store_adapter.cc][LINE:973]
[SubGraphOpt][Compile][ProcFailedCompTask] Thread[255084574985504] recompile single op[Conv_90] failed[FUNC:ProcessAllFailedCompileTasks][FILE:tbe_op_store_adapter.cc][LINE:973]
[SubGraphOpt][Compile][ProcFailedCompTask] Thread[255084574985504] recompile single op[Conv_93] failed[FUNC:ProcessAllFailedCompileTasks][FILE:tbe_op_store_adapter.cc][LINE:973]
[SubGraphOpt][Compile][ProcFailedCompTask] Thread[255084574985504] recompile single op[Conv_6] failed[FUNC:ProcessAllFailedCompileTasks][FILE:tbe_op_store_adapter.cc][LINE:973]
[SubGraphOpt][Compile][ProcFailedCompTask] Thread[255084574985504] recompile single op[Conv_16] failed[FUNC:ProcessAllFailedCompileTasks][FILE:tbe_op_store_adapter.cc][LINE:973]
[SubGraphOpt][Compile][ProcFailedCompTask] Thread[255084574985504] recompile single op[Conv_36] failed[FUNC:ProcessAllFailedCompileTasks][FILE:tbe_op_store_adapter.cc][LINE:973]
[SubGraphOpt][Compile][ProcFailedCompTask] Thread[255084574985504] recompile single op[Conv_29] failed[FUNC:ProcessAllFailedCompileTasks][FILE:tbe_op_store_adapter.cc][LINE:973]
[SubGraphOpt][Compile][ProcFailedCompTask] Thread[255084574985504] recompile single op[Conv_32] failed[FUNC:ProcessAllFailedCompileTasks][FILE:tbe_op_store_adapter.cc][LINE:973]
[SubGraphOpt][Compile][ProcFailedCompTask] Thread[255084574985504] recompile single op[Conv_39] failed[FUNC:ProcessAllFailedCompileTasks][FILE:tbe_op_store_adapter.cc][LINE:973]
[SubGraphOpt][Compile][ProcFailedCompTask] Thread[255084574985504] recompile single op[Conv_23] failed[FUNC:ProcessAllFailedCompileTasks][FILE:tbe_op_store_adapter.cc][LINE:973]
[SubGraphOpt][Compile][ProcFailedCompTask] Thread[255084574985504] recompile single op[Conv_111] failed[FUNC:ProcessAllFailedCompileTasks][FILE:tbe_op_store_adapter.cc][LINE:973]
[SubGraphOpt][Compile][ProcFailedCompTask] Thread[255084574985504] recompile single op[Conv_114] failed[FUNC:ProcessAllFailedCompileTasks][FILE:tbe_op_store_adapter.cc][LINE:973]
[SubGraphOpt][Compile][ProcFailedCompTask] Thread[255084574985504] recompile single op[Conv_104] failed[FUNC:ProcessAllFailedCompileTasks][FILE:tbe_op_store_adapter.cc][LINE:973]
[SubGraphOpt][Compile][ProcFailedCompTask] Thread[255084574985504] recompile single op[Conv_87] failed[FUNC:ProcessAllFailedCompileTasks][FILE:tbe_op_store_adapter.cc][LINE:973]
[SubGraphOpt][Compile][ProcFailedCompTask] Thread[255084574985504] recompile single op[Conv_97] failed[FUNC:ProcessAllFailedCompileTasks][FILE:tbe_op_store_adapter.cc][LINE:973]
[SubGraphOpt][Compile][ProcFailedCompTask] Thread[255084574985504] recompile single op[Conv_9] failed[FUNC:ProcessAllFailedCompileTasks][FILE:tbe_op_store_adapter.cc][LINE:973]
[SubGraphOpt][Compile][ProcFailedCompTask] Thread[255084574985504] recompile single op[Conv_47] failed[FUNC:ProcessAllFailedCompileTasks][FILE:tbe_op_store_adapter.cc][LINE:973]
[SubGraphOpt][Compile][ProcFailedCompTask] Thread[255084574985504] recompile single op[Conv_50] failed[FUNC:ProcessAllFailedCompileTasks][FILE:tbe_op_store_adapter.cc][LINE:973]
[SubGraphOpt][Compile][ProcFailedCompTask] Thread[255084574985504] recompile single op[Conv_84] failed[FUNC:ProcessAllFailedCompileTasks][FILE:tbe_op_store_adapter.cc][LINE:973]
[SubGraphOpt][Compile][ProcFailedCompTask] Thread[255084574985504] recompile single op[Conv_12] failed[FUNC:ProcessAllFailedCompileTasks][FILE:tbe_op_store_adapter.cc][LINE:973]
[SubGraphOpt][Compile][ProcFailedCompTask] Thread[255084574985504] recompile single op[Conv_77] failed[FUNC:ProcessAllFailedCompileTasks][FILE:tbe_op_store_adapter.cc][LINE:973]
[SubGraphOpt][Compile][ProcFailedCompTask] Thread[255084574985504] recompile single op[Conv_53] failed[FUNC:ProcessAllFailedCompileTasks][FILE:tbe_op_store_adapter.cc][LINE:973]
[SubGraphOpt][Compile][ProcFailedCompTask] Thread[255084574985504] recompile single op[Conv_59] failed[FUNC:ProcessAllFailedCompileTasks][FILE:tbe_op_store_adapter.cc][LINE:973]
[SubGraphOpt][Compile][ProcFailedCompTask] Thread[255084574985504] recompile single op[Conv_66] failed[FUNC:ProcessAllFailedCompileTasks][FILE:tbe_op_store_adapter.cc][LINE:973]
[SubGraphOpt][Compile][ProcFailedCompTask] Thread[255084574985504] recompile single op[Conv_73] failed[FUNC:ProcessAllFailedCompileTasks][FILE:tbe_op_store_adapter.cc][LINE:973]
[SubGraphOpt][Compile][ProcFailedCompTask] Thread[255084574985504] recompile single op[Conv_56] failed[FUNC:ProcessAllFailedCompileTasks][FILE:tbe_op_store_adapter.cc][LINE:973]
[SubGraphOpt][Compile][ProcFailedCompTask] Thread[255084574985504] recompile single op[Conv_63] failed[FUNC:ProcessAllFailedCompileTasks][FILE:tbe_op_store_adapter.cc][LINE:973]
[SubGraphOpt][Compile][ProcFailedCompTask] Thread[255084574985504] recompile single op[Conv_70] failed[FUNC:ProcessAllFailedCompileTasks][FILE:tbe_op_store_adapter.cc][LINE:973]
[SubGraphOpt][Compile][ProcFailedCompTask] Thread[255084574985504] recompile single op[Conv_81] failed[FUNC:ProcessAllFailedCompileTasks][FILE:tbe_op_store_adapter.cc][LINE:973]
[SubGraphOpt][Compile][ParalCompOp] Thread[255084574985504] process fail task failed[FUNC:ParallelCompileOp][FILE:tbe_op_store_adapter.cc][LINE:1019]
Call OptimizeFusedGraph failed, ret:-1, engine_name:AIcoreEngine, graph_name:partition0_rank1_new_sub_graph1[FUNC:OptimizeSubGraph][FILE:graph_optimize.cc][LINE:131]
subgraph 0 optimize failed[FUNC:OptimizeSubGraphWithMultiThreads][FILE:graph_manager.cc][LINE:991]
build graph failed, graph id:0, ret:-1[FUNC:BuildModelWithGraphId][FILE:ge_generator.cc][LINE:1615]
GenerateOfflineModel execute failed.
Compile operator failed, cause: 32Broken pipe File "/usr/local/Ascend/ascend-toolkit/latest/python/site-packages/tbe/common/repository_manager/interface.py", line 42, in cann_kb_finalize
RouteServer.finalize()
File "/usr/local/Ascend/ascend-toolkit/latest/python/site-packages/tbe/common/repository_manager/route.py", line 51, in wrapper
return func(cls, *args, **kwargs)
File "/usr/local/Ascend/ascend-toolkit/latest/python/site-packages/tbe/common/repository_manager/route.py", line 212, in finalize
cls.cann_kb_mgr.finalize()
File "/usr/local/Ascend/ascend-toolkit/latest/python/site-packages/tbe/common/repository_manager/cann_kb_manager/knowledge_bank_manager.py", line 41, in finalize
self.__dynamic_mgr.close()
File "/usr/local/Ascend/ascend-toolkit/latest/python/site-packages/tbe/common/repository_manager/cann_kb_manager/base_manager.py", line 325, in close
for name in copy.deepcopy(self.write_kb.keys()):
File "<string>", line 2, in keys
File "/usr/local/miniconda3/lib/python3.9/multiprocessing/managers.py", line 808, in _callmethod
conn.send((self._id, methodname, args, kwds))
File "/usr/local/miniconda3/lib/python3.9/multiprocessing/connection.py", line 211, in send
self._send_bytes(_ForkingPickler.dumps(obj))
File "/usr/local/miniconda3/lib/python3.9/multiprocessing/connection.py", line 416, in _send_bytes
self._send(header + buf)
File "/usr/local/miniconda3/lib/python3.9/multiprocessing/connection.py", line 373, in _send
n = write(self._handle, buf)
onnx导出om模型 Success