在复现官网中的智能车案例时，转换训练得到的lfnet.onnx模型失败，以下是具体的错误原因
Exception in thread Thread-1: Traceback (most recent call last): File "/usr/local/miniconda3/lib/python3.9/threading.py", line 954, in _bootstrap_inner self.run() File "/usr/local/Ascend/ascend-toolkit/latest/python/site-packages/tbe/common/repository_manager/utils/multiprocess_util.py", line 91, in run key, func, args, kwargs = self.task_q.get(timeout=TIMEOUT) File "<string>", line 2, in get File "/usr/local/miniconda3/lib/python3.9/multiprocessing/managers.py", line 809, in _callmethod kind, result = conn.recv() File "/usr/local/miniconda3/lib/python3.9/multiprocessing/connection.py", line 255, in recv buf = self._recv_bytes() File "/usr/local/miniconda3/lib/python3.9/multiprocessing/connection.py", line 419, in _recv_bytes buf = self._recv(4) File "/usr/local/miniconda3/lib/python3.9/multiprocessing/connection.py", line 388, in _recv raise EOFError EOFError Traceback (most recent call last): File "/usr/local/miniconda3/lib/python3.9/multiprocessing/util.py", line 300, in _run_finalizers finalizer() File "/usr/local/miniconda3/lib/python3.9/multiprocessing/util.py", line 224, in __call__ res = self._callback(*self._args, **self._kwargs) File "/usr/local/Ascend/ascend-toolkit/latest/python/site-packages/tbe/common/repository_manager/route.py", line 51, in wrapper return func(cls, *args, **kwargs) File "/usr/local/Ascend/ascend-toolkit/latest/python/site-packages/tbe/common/repository_manager/route.py", line 207, in finalize cls.bank_process_pool.close() AttributeError: 'NoneType' object has no attribute 'close' ... ATC run failed, Please check the detail log, Try 'atc --help' for more information E61001: In op [fix_pipe], [fix_pipe does not support single op compilation!] TraceBack (most recent call last): Failed to compile Op [Conv_41function_graph_15/Conv_41_pass.1,[Relu_42,Relu_42]]. (oppath: [Compile /usr/local/Ascend/ascend-toolkit/7.0.RC1/opp/built-in/op_impl/ai_core/tbe/impl/fix_pipe.py failed with errormsg/stack: File "/usr/local/Ascend/ascend-toolkit/latest/python/site-packages/tbe/common/utils/errormgr/error_manager_vector.py", line 284, in raise_err_specific_reson raise RuntimeError(args_dict, msg) RuntimeError: ({'errCode': 'E61001', 'op_name': 'fix_pipe', 'reason': 'fix_pipe does not support single op compilation!'}, 'In op [fix_pipe], [fix_pipe does not support single op compilation!]') ], optype: [FixPipe]) Compile op[Conv_41function_graph_15/Conv_41_pass.1,[Relu_42,Relu_42]] failed, oppath[/usr/local/Ascend/ascend-toolkit/7.0.RC1/opp/built-in/op_impl/ai_core/tbe/impl/fix_pipe.py], optype[FixPipe], taskID[77]. Please check op's compilation error message.[FUNC:ReportBuildErrMessage][FILE:fusion_manager.cc][LINE:763] Failed to compile Op [Conv_43function_graph_16/Conv_43_pass.2,[Relu_45,Relu_45,Add_44]]. (oppath: [Compile /usr/local/Ascend/ascend-toolkit/7.0.RC1/opp/built-in/op_impl/ai_core/tbe/impl/fix_pipe.py failed with errormsg/stack: File "/usr/local/Ascend/ascend-toolkit/latest/python/site-packages/tbe/common/utils/errormgr/error_manager_vector.py", line 284, in raise_err_specific_reson raise RuntimeError(args_dict, msg) RuntimeError: ({'errCode': 'E61001', 'op_name': 'fix_pipe', 'reason': 'fix_pipe does not support single op compilation!'}, 'In op [fix_pipe], [fix_pipe does not support single op compilation!]') ], optype: [FixPipe]) Compile op[Conv_43function_graph_16/Conv_43_pass.2,[Relu_45,Relu_45,Add_44]] failed, oppath[/usr/local/Ascend/ascend-toolkit/7.0.RC1/opp/built-in/op_impl/ai_core/tbe/impl/fix_pipe.py], optype[FixPipe], taskID[79]. Please check op's compilation error message.[FUNC:ReportBuildErrMessage][FILE:fusion_manager.cc][LINE:763] Failed to compile Op [Conv_27function_graph_10/Conv_27_pass.2,[Relu_29,Relu_29,Add_28]]. (oppath: [Compile /usr/local/Ascend/ascend-toolkit/7.0.RC1/opp/built-in/op_impl/ai_core/tbe/impl/fix_pipe.py failed with errormsg/stack: File "/usr/local/Ascend/ascend-toolkit/latest/python/site-packages/tbe/common/utils/errormgr/error_manager_vector.py", line 284, in raise_err_specific_reson raise RuntimeError(args_dict, msg) RuntimeError: ({'errCode': 'E61001', 'op_name': 'fix_pipe', 'reason': 'fix_pipe does not support single op compilation!'}, 'In op [fix_pipe], [fix_pipe does not support single op compilation!]') ], optype: [FixPipe]) Compile op[Conv_27function_graph_10/Conv_27_pass.2,[Relu_29,Relu_29,Add_28]] failed, oppath[/usr/local/Ascend/ascend-toolkit/7.0.RC1/opp/built-in/op_impl/ai_core/tbe/impl/fix_pipe.py], optype[FixPipe], taskID[89]. Please check op's compilation error message.[FUNC:ReportBuildErrMessage][FILE:fusion_manager.cc][LINE:763] Failed to compile Op [Conv_30function_graph_11/Conv_30_pass.1,[Relu_31,Relu_31]]. (oppath: [Compile /usr/local/Ascend/ascend-toolkit/7.0.RC1/opp/built-in/op_impl/ai_core/tbe/impl/fix_pipe.py failed with errormsg/stack: File "/usr/local/Ascend/ascend-toolkit/latest/python/site-packages/tbe/common/utils/errormgr/error_manager_vector.py", line 284, in raise_err_specific_reson raise RuntimeError(args_dict, msg) RuntimeError: ({'errCode': 'E61001', 'op_name': 'fix_pipe', 'reason': 'fix_pipe does not support single op compilation!'}, 'In op [fix_pipe], [fix_pipe does not support single op compilation!]') ], optype: [FixPipe]) Compile op[Conv_30function_graph_11/Conv_30_pass.1,[Relu_31,Relu_31]] failed, oppath[/usr/local/Ascend/ascend-toolkit/7.0.RC1/opp/built-in/op_impl/ai_core/tbe/impl/fix_pipe.py], optype[FixPipe], taskID[87]. Please check op's compilation error message.[FUNC:ReportBuildErrMessage][FILE:fusion_manager.cc][LINE:763] Failed to compile Op [Conv_35function_graph_13/Conv_35,[Conv_35,Conv_35]]. (oppath: [Compile /usr/local/Ascend/ascend-toolkit/7.0.RC1/opp/built-in/op_impl/ai_core/tbe/impl/conv2d.py failed with errormsg/stack: File "/usr/local/miniconda3/lib/python3.9/multiprocessing/connection.py", line 635, in SocketClient s.connect(address) ConnectionRefusedError: [Errno 111] Connection refused ], optype: [Conv2D]) Compile op[Conv_35function_graph_13/Conv_35,[Conv_35,Conv_35]] failed, oppath[/usr/local/Ascend/ascend-toolkit/7.0.RC1/opp/built-in/op_impl/ai_core/tbe/impl/conv2d.py], optype[Conv2D], taskID[82]. Please check op's compilation error message.[FUNC:ReportBuildErrMessage][FILE:fusion_manager.cc][LINE:763] Failed to compile Op [Conv_38function_graph_14/Conv_38_pass.2,[Relu_40,Relu_40,Add_39]]. (oppath: [Compile /usr/local/Ascend/ascend-toolkit/7.0.RC1/opp/built-in/op_impl/ai_core/tbe/impl/fix_pipe.py failed with errormsg/stack: File "/usr/local/Ascend/ascend-toolkit/latest/python/site-packages/tbe/common/utils/errormgr/error_manager_vector.py", line 284, in raise_err_specific_reson raise RuntimeError(args_dict, msg) RuntimeError: ({'errCode': 'E61001', 'op_name': 'fix_pipe', 'reason': 'fix_pipe does not support single op compilation!'}, 'In op [fix_pipe], [fix_pipe does not support single op compilation!]') ], optype: [FixPipe]) Compile op[Conv_38function_graph_14/Conv_38_pass.2,[Relu_40,Relu_40,Add_39]] failed, oppath[/usr/local/Ascend/ascend-toolkit/7.0.RC1/opp/built-in/op_impl/ai_core/tbe/impl/fix_pipe.py], optype[FixPipe], taskID[81]. Please check op's compilation error message.[FUNC:ReportBuildErrMessage][FILE:fusion_manager.cc][LINE:763] Failed to compile Op [Conv_32function_graph_12/Conv_32_pass.2,[Relu_34,Relu_34,Add_33]]. (oppath: [Compile /usr/local/Ascend/ascend-toolkit/7.0.RC1/opp/built-in/op_impl/ai_core/tbe/impl/fix_pipe.py failed with errormsg/stack: File "/usr/local/Ascend/ascend-toolkit/latest/python/site-packages/tbe/common/utils/errormgr/error_manager_vector.py", line 284, in raise_err_specific_reson raise RuntimeError(args_dict, msg) RuntimeError: ({'errCode': 'E61001', 'op_name': 'fix_pipe', 'reason': 'fix_pipe does not support single op compilation!'}, 'In op [fix_pipe], [fix_pipe does not support single op compilation!]') ], optype: [FixPipe]) Compile op[Conv_32function_graph_12/Conv_32_pass.2,[Relu_34,Relu_34,Add_33]] failed, oppath[/usr/local/Ascend/ascend-toolkit/7.0.RC1/opp/built-in/op_impl/ai_core/tbe/impl/fix_pipe.py], optype[FixPipe], taskID[85]. Please check op's compilation error message.[FUNC:ReportBuildErrMessage][FILE:fusion_manager.cc][LINE:763] Failed to compile Op [Conv_27function_graph_10/Conv_27,[Conv_27,Conv_27]]. (oppath: [Compile /usr/local/Ascend/ascend-toolkit/7.0.RC1/opp/built-in/op_impl/ai_core/tbe/impl/conv2d.py failed with errormsg/stack: File "/usr/local/miniconda3/lib/python3.9/multiprocessing/connection.py", line 635, in SocketClient s.connect(address) ConnectionRefusedError: [Errno 111] Connection refused ], optype: [Conv2D]) Compile op[Conv_27function_graph_10/Conv_27,[Conv_27,Conv_27]] failed, oppath[/usr/local/Ascend/ascend-toolkit/7.0.RC1/opp/built-in/op_impl/ai_core/tbe/impl/conv2d.py], optype[Conv2D], taskID[88]. Please check op's compilation error message.[FUNC:ReportBuildErrMessage][FILE:fusion_manager.cc][LINE:763] Failed to compile Op [Conv_38function_graph_14/Conv_38,[Conv_38,Conv_38]]. (oppath: [Compile /usr/local/Ascend/ascend-toolkit/7.0.RC1/opp/built-in/op_impl/ai_core/tbe/impl/conv2d.py failed with errormsg/stack: File "/usr/local/miniconda3/lib/python3.9/multiprocessing/connection.py", line 635, in SocketClient s.connect(address) ConnectionRefusedError: [Errno 111] Connection refused ], optype: [Conv2D]) Compile op[Conv_38function_graph_14/Conv_38,[Conv_38,Conv_38]] failed, oppath[/usr/local/Ascend/ascend-toolkit/7.0.RC1/opp/built-in/op_impl/ai_core/tbe/impl/conv2d.py], optype[Conv2D], taskID[80]. Please check op's compilation error message.[FUNC:ReportBuildErrMessage][FILE:fusion_manager.cc][LINE:763] [SubGraphOpt][Compile][ProcFailedCompTask] Thread[255084902075680] recompile single op[Conv_38function_graph_14/Conv_38] failed[FUNC:ProcessAllFailedCompileTasks][FILE:tbe_op_store_adapter.cc][LINE:973] [SubGraphOpt][Compile][ProcFailedCompTask] Thread[255084902075680] recompile single op[Conv_27function_graph_10/Conv_27] failed[FUNC:ProcessAllFailedCompileTasks][FILE:tbe_op_store_adapter.cc][LINE:973] [SubGraphOpt][Compile][ProcFailedCompTask] Thread[255084902075680] recompile single op[Conv_32function_graph_12/Conv_32_pass.2] failed[FUNC:ProcessAllFailedCompileTasks][FILE:tbe_op_store_adapter.cc][LINE:973] [SubGraphOpt][Compile][ProcFailedCompTask] Thread[255084902075680] recompile single op[Conv_38function_graph_14/Conv_38_pass.2] failed[FUNC:ProcessAllFailedCompileTasks][FILE:tbe_op_store_adapter.cc][LINE:973] [SubGraphOpt][Compile][ProcFailedCompTask] Thread[255084902075680] recompile single op[Conv_35function_graph_13/Conv_35] failed[FUNC:ProcessAllFailedCompileTasks][FILE:tbe_op_store_adapter.cc][LINE:973] [SubGraphOpt][Compile][ProcFailedCompTask] Thread[255084902075680] recompile single op[Conv_30function_graph_11/Conv_30_pass.1] failed[FUNC:ProcessAllFailedCompileTasks][FILE:tbe_op_store_adapter.cc][LINE:973] [SubGraphOpt][Compile][ProcFailedCompTask] Thread[255084902075680] recompile single op[Conv_27function_graph_10/Conv_27_pass.2] failed[FUNC:ProcessAllFailedCompileTasks][FILE:tbe_op_store_adapter.cc][LINE:973] [SubGraphOpt][Compile][ProcFailedCompTask] Thread[255084902075680] recompile single op[Conv_43function_graph_16/Conv_43_pass.2] failed[FUNC:ProcessAllFailedCompileTasks][FILE:tbe_op_store_adapter.cc][LINE:973] [SubGraphOpt][Compile][ProcFailedCompTask] Thread[255084902075680] recompile single op[Conv_35function_graph_13/Conv_35_pass.1] failed[FUNC:ProcessAllFailedCompileTasks][FILE:tbe_op_store_adapter.cc][LINE:973] [SubGraphOpt][Compile][ProcFailedCompTask] Thread[255084902075680] recompile single op[Conv_41function_graph_15/Conv_41_pass.1] failed[FUNC:ProcessAllFailedCompileTasks][FILE:tbe_op_store_adapter.cc][LINE:973] [SubGraphOpt][Compile][ParalCompOp] Thread[255084902075680] process fail task failed[FUNC:ParallelCompileOp][FILE:tbe_op_store_adapter.cc][LINE:1019] [SubGraphOpt][Compile][CompOpOnly] CompileOp failed.[FUNC:CompileOpOnly][FILE:op_compiler.cc][LINE:1129] [GraphOpt][FusedGraph][RunCompile] Failed to compile graph with compiler Normal mode Op Compiler[FUNC:SubGraphCompile][FILE:fe_graph_optimizer.cc][LINE:1291] Call OptimizeFusedGraph failed, ret:-1, engine_name:AIcoreEngine, graph_name:partition0_rank1_new_sub_graph1[FUNC:OptimizeSubGraph][FILE:graph_optimize.cc][LINE:131] subgraph 0 optimize failed[FUNC:OptimizeSubGraphWithMultiThreads][FILE:graph_manager.cc][LINE:991] build graph failed, graph id:0, ret:-1[FUNC:BuildModelWithGraphId][FILE:ge_generator.cc][LINE:1615] GenerateOfflineModel execute failed. Compile operator failed, cause: 32Broken pipe File "/usr/local/Ascend/ascend-toolkit/latest/python/site-packages/tbe/common/repository_manager/interface.py", line 42, in cann_kb_finalize RouteServer.finalize() File "/usr/local/Ascend/ascend-toolkit/latest/python/site-packages/tbe/common/repository_manager/route.py", line 51, in wrapper return func(cls, *args, **kwargs) File "/usr/local/Ascend/ascend-toolkit/latest/python/site-packages/tbe/common/repository_manager/route.py", line 212, in finalize cls.cann_kb_mgr.finalize() File "/usr/local/Ascend/ascend-toolkit/latest/python/site-packages/tbe/common/repository_manager/cann_kb_manager/knowledge_bank_manager.py", line 41, in finalize self.__dynamic_mgr.close() File "/usr/local/Ascend/ascend-toolkit/latest/python/site-packages/tbe/common/repository_manager/cann_kb_manager/base_manager.py", line 325, in close for name in copy.deepcopy(self.write_kb.keys()): File "<string>", line 2, in keys File "/usr/local/miniconda3/lib/python3.9/multiprocessing/managers.py", line 808, in _callmethod conn.send((self._id, methodname, args, kwds)) File "/usr/local/miniconda3/lib/python3.9/multiprocessing/connection.py", line 211, in send self._send_bytes(_ForkingPickler.dumps(obj)) File "/usr/local/miniconda3/lib/python3.9/multiprocessing/connection.py", line 416, in _send_bytes self._send(header + buf) File "/usr/local/miniconda3/lib/python3.9/multiprocessing/connection.py", line 373, in _send n = write(self._handle, buf)
点赞
收藏
回复
分享
举报
在复现官网中的智能车案例时，转换训练得到的lfnet.onnx模型失败，以下是具体的错误原因
Exception in thread Thread-1: Traceback (most recent call last): File "/usr/local/miniconda3/lib/python3.9/threading.py", line 954, in _bootstrap_inner self.run() File "/usr/local/Ascend/ascend-toolkit/latest/python/site-packages/tbe/common/repository_manager/utils/multiprocess_util.py", line 91, in run key, func, args, kwargs = self.task_q.get(timeout=TIMEOUT) File "<string>", line 2, in get File "/usr/local/miniconda3/lib/python3.9/multiprocessing/managers.py", line 809, in _callmethod kind, result = conn.recv() File "/usr/local/miniconda3/lib/python3.9/multiprocessing/connection.py", line 255, in recv buf = self._recv_bytes() File "/usr/local/miniconda3/lib/python3.9/multiprocessing/connection.py", line 419, in _recv_bytes buf = self._recv(4) File "/usr/local/miniconda3/lib/python3.9/multiprocessing/connection.py", line 388, in _recv raise EOFError EOFError Traceback (most recent call last): File "/usr/local/miniconda3/lib/python3.9/multiprocessing/util.py", line 300, in _run_finalizers finalizer() File "/usr/local/miniconda3/lib/python3.9/multiprocessing/util.py", line 224, in __call__ res = self._callback(*self._args, **self._kwargs) File "/usr/local/Ascend/ascend-toolkit/latest/python/site-packages/tbe/common/repository_manager/route.py", line 51, in wrapper return func(cls, *args, **kwargs) File "/usr/local/Ascend/ascend-toolkit/latest/python/site-packages/tbe/common/repository_manager/route.py", line 207, in finalize cls.bank_process_pool.close() AttributeError: 'NoneType' object has no attribute 'close' ... ATC run failed, Please check the detail log, Try 'atc --help' for more information E61001: In op [fix_pipe], [fix_pipe does not support single op compilation!] TraceBack (most recent call last): Failed to compile Op [Conv_41function_graph_15/Conv_41_pass.1,[Relu_42,Relu_42]]. (oppath: [Compile /usr/local/Ascend/ascend-toolkit/7.0.RC1/opp/built-in/op_impl/ai_core/tbe/impl/fix_pipe.py failed with errormsg/stack: File "/usr/local/Ascend/ascend-toolkit/latest/python/site-packages/tbe/common/utils/errormgr/error_manager_vector.py", line 284, in raise_err_specific_reson raise RuntimeError(args_dict, msg) RuntimeError: ({'errCode': 'E61001', 'op_name': 'fix_pipe', 'reason': 'fix_pipe does not support single op compilation!'}, 'In op [fix_pipe], [fix_pipe does not support single op compilation!]') ], optype: [FixPipe]) Compile op[Conv_41function_graph_15/Conv_41_pass.1,[Relu_42,Relu_42]] failed, oppath[/usr/local/Ascend/ascend-toolkit/7.0.RC1/opp/built-in/op_impl/ai_core/tbe/impl/fix_pipe.py], optype[FixPipe], taskID[77]. Please check op's compilation error message.[FUNC:ReportBuildErrMessage][FILE:fusion_manager.cc][LINE:763] Failed to compile Op [Conv_43function_graph_16/Conv_43_pass.2,[Relu_45,Relu_45,Add_44]]. (oppath: [Compile /usr/local/Ascend/ascend-toolkit/7.0.RC1/opp/built-in/op_impl/ai_core/tbe/impl/fix_pipe.py failed with errormsg/stack: File "/usr/local/Ascend/ascend-toolkit/latest/python/site-packages/tbe/common/utils/errormgr/error_manager_vector.py", line 284, in raise_err_specific_reson raise RuntimeError(args_dict, msg) RuntimeError: ({'errCode': 'E61001', 'op_name': 'fix_pipe', 'reason': 'fix_pipe does not support single op compilation!'}, 'In op [fix_pipe], [fix_pipe does not support single op compilation!]') ], optype: [FixPipe]) Compile op[Conv_43function_graph_16/Conv_43_pass.2,[Relu_45,Relu_45,Add_44]] failed, oppath[/usr/local/Ascend/ascend-toolkit/7.0.RC1/opp/built-in/op_impl/ai_core/tbe/impl/fix_pipe.py], optype[FixPipe], taskID[79]. Please check op's compilation error message.[FUNC:ReportBuildErrMessage][FILE:fusion_manager.cc][LINE:763] Failed to compile Op [Conv_27function_graph_10/Conv_27_pass.2,[Relu_29,Relu_29,Add_28]]. (oppath: [Compile /usr/local/Ascend/ascend-toolkit/7.0.RC1/opp/built-in/op_impl/ai_core/tbe/impl/fix_pipe.py failed with errormsg/stack: File "/usr/local/Ascend/ascend-toolkit/latest/python/site-packages/tbe/common/utils/errormgr/error_manager_vector.py", line 284, in raise_err_specific_reson raise RuntimeError(args_dict, msg) RuntimeError: ({'errCode': 'E61001', 'op_name': 'fix_pipe', 'reason': 'fix_pipe does not support single op compilation!'}, 'In op [fix_pipe], [fix_pipe does not support single op compilation!]') ], optype: [FixPipe]) Compile op[Conv_27function_graph_10/Conv_27_pass.2,[Relu_29,Relu_29,Add_28]] failed, oppath[/usr/local/Ascend/ascend-toolkit/7.0.RC1/opp/built-in/op_impl/ai_core/tbe/impl/fix_pipe.py], optype[FixPipe], taskID[89]. Please check op's compilation error message.[FUNC:ReportBuildErrMessage][FILE:fusion_manager.cc][LINE:763] Failed to compile Op [Conv_30function_graph_11/Conv_30_pass.1,[Relu_31,Relu_31]]. (oppath: [Compile /usr/local/Ascend/ascend-toolkit/7.0.RC1/opp/built-in/op_impl/ai_core/tbe/impl/fix_pipe.py failed with errormsg/stack: File "/usr/local/Ascend/ascend-toolkit/latest/python/site-packages/tbe/common/utils/errormgr/error_manager_vector.py", line 284, in raise_err_specific_reson raise RuntimeError(args_dict, msg) RuntimeError: ({'errCode': 'E61001', 'op_name': 'fix_pipe', 'reason': 'fix_pipe does not support single op compilation!'}, 'In op [fix_pipe], [fix_pipe does not support single op compilation!]') ], optype: [FixPipe]) Compile op[Conv_30function_graph_11/Conv_30_pass.1,[Relu_31,Relu_31]] failed, oppath[/usr/local/Ascend/ascend-toolkit/7.0.RC1/opp/built-in/op_impl/ai_core/tbe/impl/fix_pipe.py], optype[FixPipe], taskID[87]. Please check op's compilation error message.[FUNC:ReportBuildErrMessage][FILE:fusion_manager.cc][LINE:763] Failed to compile Op [Conv_35function_graph_13/Conv_35,[Conv_35,Conv_35]]. (oppath: [Compile /usr/local/Ascend/ascend-toolkit/7.0.RC1/opp/built-in/op_impl/ai_core/tbe/impl/conv2d.py failed with errormsg/stack: File "/usr/local/miniconda3/lib/python3.9/multiprocessing/connection.py", line 635, in SocketClient s.connect(address) ConnectionRefusedError: [Errno 111] Connection refused ], optype: [Conv2D]) Compile op[Conv_35function_graph_13/Conv_35,[Conv_35,Conv_35]] failed, oppath[/usr/local/Ascend/ascend-toolkit/7.0.RC1/opp/built-in/op_impl/ai_core/tbe/impl/conv2d.py], optype[Conv2D], taskID[82]. Please check op's compilation error message.[FUNC:ReportBuildErrMessage][FILE:fusion_manager.cc][LINE:763] Failed to compile Op [Conv_38function_graph_14/Conv_38_pass.2,[Relu_40,Relu_40,Add_39]]. (oppath: [Compile /usr/local/Ascend/ascend-toolkit/7.0.RC1/opp/built-in/op_impl/ai_core/tbe/impl/fix_pipe.py failed with errormsg/stack: File "/usr/local/Ascend/ascend-toolkit/latest/python/site-packages/tbe/common/utils/errormgr/error_manager_vector.py", line 284, in raise_err_specific_reson raise RuntimeError(args_dict, msg) RuntimeError: ({'errCode': 'E61001', 'op_name': 'fix_pipe', 'reason': 'fix_pipe does not support single op compilation!'}, 'In op [fix_pipe], [fix_pipe does not support single op compilation!]') ], optype: [FixPipe]) Compile op[Conv_38function_graph_14/Conv_38_pass.2,[Relu_40,Relu_40,Add_39]] failed, oppath[/usr/local/Ascend/ascend-toolkit/7.0.RC1/opp/built-in/op_impl/ai_core/tbe/impl/fix_pipe.py], optype[FixPipe], taskID[81]. Please check op's compilation error message.[FUNC:ReportBuildErrMessage][FILE:fusion_manager.cc][LINE:763] Failed to compile Op [Conv_32function_graph_12/Conv_32_pass.2,[Relu_34,Relu_34,Add_33]]. (oppath: [Compile /usr/local/Ascend/ascend-toolkit/7.0.RC1/opp/built-in/op_impl/ai_core/tbe/impl/fix_pipe.py failed with errormsg/stack: File "/usr/local/Ascend/ascend-toolkit/latest/python/site-packages/tbe/common/utils/errormgr/error_manager_vector.py", line 284, in raise_err_specific_reson raise RuntimeError(args_dict, msg) RuntimeError: ({'errCode': 'E61001', 'op_name': 'fix_pipe', 'reason': 'fix_pipe does not support single op compilation!'}, 'In op [fix_pipe], [fix_pipe does not support single op compilation!]') ], optype: [FixPipe]) Compile op[Conv_32function_graph_12/Conv_32_pass.2,[Relu_34,Relu_34,Add_33]] failed, oppath[/usr/local/Ascend/ascend-toolkit/7.0.RC1/opp/built-in/op_impl/ai_core/tbe/impl/fix_pipe.py], optype[FixPipe], taskID[85]. Please check op's compilation error message.[FUNC:ReportBuildErrMessage][FILE:fusion_manager.cc][LINE:763] Failed to compile Op [Conv_27function_graph_10/Conv_27,[Conv_27,Conv_27]]. (oppath: [Compile /usr/local/Ascend/ascend-toolkit/7.0.RC1/opp/built-in/op_impl/ai_core/tbe/impl/conv2d.py failed with errormsg/stack: File "/usr/local/miniconda3/lib/python3.9/multiprocessing/connection.py", line 635, in SocketClient s.connect(address) ConnectionRefusedError: [Errno 111] Connection refused ], optype: [Conv2D]) Compile op[Conv_27function_graph_10/Conv_27,[Conv_27,Conv_27]] failed, oppath[/usr/local/Ascend/ascend-toolkit/7.0.RC1/opp/built-in/op_impl/ai_core/tbe/impl/conv2d.py], optype[Conv2D], taskID[88]. Please check op's compilation error message.[FUNC:ReportBuildErrMessage][FILE:fusion_manager.cc][LINE:763] Failed to compile Op [Conv_38function_graph_14/Conv_38,[Conv_38,Conv_38]]. (oppath: [Compile /usr/local/Ascend/ascend-toolkit/7.0.RC1/opp/built-in/op_impl/ai_core/tbe/impl/conv2d.py failed with errormsg/stack: File "/usr/local/miniconda3/lib/python3.9/multiprocessing/connection.py", line 635, in SocketClient s.connect(address) ConnectionRefusedError: [Errno 111] Connection refused ], optype: [Conv2D]) Compile op[Conv_38function_graph_14/Conv_38,[Conv_38,Conv_38]] failed, oppath[/usr/local/Ascend/ascend-toolkit/7.0.RC1/opp/built-in/op_impl/ai_core/tbe/impl/conv2d.py], optype[Conv2D], taskID[80]. Please check op's compilation error message.[FUNC:ReportBuildErrMessage][FILE:fusion_manager.cc][LINE:763] [SubGraphOpt][Compile][ProcFailedCompTask] Thread[255084902075680] recompile single op[Conv_38function_graph_14/Conv_38] failed[FUNC:ProcessAllFailedCompileTasks][FILE:tbe_op_store_adapter.cc][LINE:973] [SubGraphOpt][Compile][ProcFailedCompTask] Thread[255084902075680] recompile single op[Conv_27function_graph_10/Conv_27] failed[FUNC:ProcessAllFailedCompileTasks][FILE:tbe_op_store_adapter.cc][LINE:973] [SubGraphOpt][Compile][ProcFailedCompTask] Thread[255084902075680] recompile single op[Conv_32function_graph_12/Conv_32_pass.2] failed[FUNC:ProcessAllFailedCompileTasks][FILE:tbe_op_store_adapter.cc][LINE:973] [SubGraphOpt][Compile][ProcFailedCompTask] Thread[255084902075680] recompile single op[Conv_38function_graph_14/Conv_38_pass.2] failed[FUNC:ProcessAllFailedCompileTasks][FILE:tbe_op_store_adapter.cc][LINE:973] [SubGraphOpt][Compile][ProcFailedCompTask] Thread[255084902075680] recompile single op[Conv_35function_graph_13/Conv_35] failed[FUNC:ProcessAllFailedCompileTasks][FILE:tbe_op_store_adapter.cc][LINE:973] [SubGraphOpt][Compile][ProcFailedCompTask] Thread[255084902075680] recompile single op[Conv_30function_graph_11/Conv_30_pass.1] failed[FUNC:ProcessAllFailedCompileTasks][FILE:tbe_op_store_adapter.cc][LINE:973] [SubGraphOpt][Compile][ProcFailedCompTask] Thread[255084902075680] recompile single op[Conv_27function_graph_10/Conv_27_pass.2] failed[FUNC:ProcessAllFailedCompileTasks][FILE:tbe_op_store_adapter.cc][LINE:973] [SubGraphOpt][Compile][ProcFailedCompTask] Thread[255084902075680] recompile single op[Conv_43function_graph_16/Conv_43_pass.2] failed[FUNC:ProcessAllFailedCompileTasks][FILE:tbe_op_store_adapter.cc][LINE:973] [SubGraphOpt][Compile][ProcFailedCompTask] Thread[255084902075680] recompile single op[Conv_35function_graph_13/Conv_35_pass.1] failed[FUNC:ProcessAllFailedCompileTasks][FILE:tbe_op_store_adapter.cc][LINE:973] [SubGraphOpt][Compile][ProcFailedCompTask] Thread[255084902075680] recompile single op[Conv_41function_graph_15/Conv_41_pass.1] failed[FUNC:ProcessAllFailedCompileTasks][FILE:tbe_op_store_adapter.cc][LINE:973] [SubGraphOpt][Compile][ParalCompOp] Thread[255084902075680] process fail task failed[FUNC:ParallelCompileOp][FILE:tbe_op_store_adapter.cc][LINE:1019] [SubGraphOpt][Compile][CompOpOnly] CompileOp failed.[FUNC:CompileOpOnly][FILE:op_compiler.cc][LINE:1129] [GraphOpt][FusedGraph][RunCompile] Failed to compile graph with compiler Normal mode Op Compiler[FUNC:SubGraphCompile][FILE:fe_graph_optimizer.cc][LINE:1291] Call OptimizeFusedGraph failed, ret:-1, engine_name:AIcoreEngine, graph_name:partition0_rank1_new_sub_graph1[FUNC:OptimizeSubGraph][FILE:graph_optimize.cc][LINE:131] subgraph 0 optimize failed[FUNC:OptimizeSubGraphWithMultiThreads][FILE:graph_manager.cc][LINE:991] build graph failed, graph id:0, ret:-1[FUNC:BuildModelWithGraphId][FILE:ge_generator.cc][LINE:1615] GenerateOfflineModel execute failed. Compile operator failed, cause: 32Broken pipe File "/usr/local/Ascend/ascend-toolkit/latest/python/site-packages/tbe/common/repository_manager/interface.py", line 42, in cann_kb_finalize RouteServer.finalize() File "/usr/local/Ascend/ascend-toolkit/latest/python/site-packages/tbe/common/repository_manager/route.py", line 51, in wrapper return func(cls, *args, **kwargs) File "/usr/local/Ascend/ascend-toolkit/latest/python/site-packages/tbe/common/repository_manager/route.py", line 212, in finalize cls.cann_kb_mgr.finalize() File "/usr/local/Ascend/ascend-toolkit/latest/python/site-packages/tbe/common/repository_manager/cann_kb_manager/knowledge_bank_manager.py", line 41, in finalize self.__dynamic_mgr.close() File "/usr/local/Ascend/ascend-toolkit/latest/python/site-packages/tbe/common/repository_manager/cann_kb_manager/base_manager.py", line 325, in close for name in copy.deepcopy(self.write_kb.keys()): File "<string>", line 2, in keys File "/usr/local/miniconda3/lib/python3.9/multiprocessing/managers.py", line 808, in _callmethod conn.send((self._id, methodname, args, kwds)) File "/usr/local/miniconda3/lib/python3.9/multiprocessing/connection.py", line 211, in send self._send_bytes(_ForkingPickler.dumps(obj)) File "/usr/local/miniconda3/lib/python3.9/multiprocessing/connection.py", line 416, in _send_bytes self._send(header + buf) File "/usr/local/miniconda3/lib/python3.9/multiprocessing/connection.py", line 373, in _send n = write(self._handle, buf)