目前尝试 转310B1 的模型 只有fp32 能转： atc --model=/app/resources/bs1_simplify.onnx --framework=5 --output=/app/resources/model_fp32 --soc_version=Ascend310B1 --log=debug --precision_mode=force_fp32
如果换成 atc --model=/app/resources/bs1_simplify.onnx --framework=5 --output=/app/resources/model_fp16 --soc_version=Ascend310B1 --log=debug --precision_mode=force_fp16，都会失败， 在300V 和910B 上都进行了尝试 都不行应该不是 设备资源不够的问题吧
报错
ATC run failed, Please check the detail log, Try 'atc --help' for more information
E40021: 2025-01-18-01:43:35.711.466 Failed to compile Op [/model/neck/fpn_blocks.0/m/m.2/conv2/conv/Convfunction_graph_45//model/neck/fpn_blocks.0/m/m.2/conv2/conv/Conv_pass.1,[/model/neck/fpn_blocks.0/m/m.2/Add,/model/neck/fpn_blocks.0/m/m.2/Add]]. (oppath: [Compile /usr/local/Ascend/ascend-toolkit/8.0.RC2.alpha001/opp/built-in/op_impl/ai_core/tbe/impl/fix_pipe.py failed with errormsg/stack: File "/usr/local/Ascend/ascend-toolkit/latest/python/site-packages/tbe/common/utils/errormgr/error_manager_vector.py", line 284, in raise_err_specific_reson
raise RuntimeError(args_dict, msg)
RuntimeError: ({'errCode': 'E61001', 'op_name': 'fix_pipe', 'reason': 'fix_pipe does not support single op compilation!'}, 'In op [fix_pipe], [fix_pipe does not support single op compilation!]')
], optype: [FixPipe])
Solution: See the host log for details, and then check the Python stack where the error log is reported.
TraceBack (most recent call last):
Compile op[/model/neck/fpn_blocks.0/m/m.2/conv2/conv/Convfunction_graph_45//model/neck/fpn_blocks.0/m/m.2/conv2/conv/Conv_pass.1,[/model/neck/fpn_blocks.0/m/m.2/Add,/model/neck/fpn_blocks.0/m/m.2/Add]] failed, oppath[/usr/local/Ascend/ascend-toolkit/8.0.RC2.alpha001/opp/built-in/op_impl/ai_core/tbe/impl/fix_pipe.py], optype[FixPipe], taskID[2364]. Please check op's compilation error message.[FUNC:ReportBuildErrMessage][FILE:fusion_manager.cc][LINE:748]
[SubGraphOpt][Compile][ProcFailedCompTask] Thread[281470438439600] recompile single op[/model/neck/fpn_blocks.0/m/m.0/conv2/conv/Convfunction_graph_43//model/neck/fpn_blocks.0/m/m.0/conv2/conv/Conv_pass.1] failed[FUNC:ProcessAllFailedCompileTasks][FILE:tbe_op_store_adapter.cc][LINE:962]
[SubGraphOpt][Compile][ProcFailedCompTask] Thread[281470438439600] recompile single op[/model/neck/fpn_blocks.0/m/m.1/conv2/conv/Convfunction_graph_44//model/neck/fpn_blocks.0/m/m.1/conv2/conv/Conv_pass.1] failed[FUNC:ProcessAllFailedCompileTasks][FILE:tbe_op_store_adapter.cc][LINE:962]
[SubGraphOpt][Compile][ProcFailedCompTask] Thread[281470438439600] recompile single op[/model/neck/fpn_blocks.0/m/m.2/conv2/conv/Convfunction_graph_45//model/neck/fpn_blocks.0/m/m.2/conv2/conv/Conv_pass.1] failed[FUNC:ProcessAllFailedCompileTasks][FILE:tbe_op_store_adapter.cc][LINE:962]
[SubGraphOpt][Compile][ParalCompOp] Thread[281470438439600] process fail task failed[FUNC:ParallelCompileOp][FILE:tbe_op_store_adapter.cc][LINE:1010]
[SubGraphOpt][Compile][CompOpOnly] CompileOp failed.[FUNC:CompileOpOnly][FILE:op_compiler.cc][LINE:1119]
[GraphOpt][FusedGraph][RunCompile] Failed to compile graph with compiler Normal mode Op Compiler[FUNC:SubGraphCompile][FILE:fe_graph_optimizer.cc][LINE:1385]
Call OptimizeFusedGraph failed, ret:-1, engine_name:AIcoreEngine, graph_name:partition0_rank120_new_sub_graph941[FUNC:OptimizeSubGraph][FILE:graph_optimize.cc][LINE:126]
subgraph 119 optimize failed[FUNC:OptimizeSubGraphWithMultiThreads][FILE:graph_manager.cc][LINE:1024]
Failed to compile Op [/model/neck/pan_blocks.1/m/m.2/conv2/conv/Convfunction_graph_54//model/neck/pan_blocks.1/m/m.2/conv2/conv/Conv_pass.1,[/model/neck/pan_blocks.1/m/m.2/Add,/model/neck/pan_blocks.1/m/m.2/Add]]. (oppath: [Compile /usr/local/Ascend/ascend-toolkit/8.0.RC2.alpha001/opp/built-in/op_impl/ai_core/tbe/impl/fix_pipe.py failed with errormsg/stack: File "/usr/local/Ascend/ascend-toolkit/latest/python/site-packages/tbe/common/utils/errormgr/error_manager_vector.py", line 284, in raise_err_specific_reson
raise RuntimeError(args_dict, msg)
RuntimeError: ({'errCode': 'E61001', 'op_name': 'fix_pipe', 'reason': 'fix_pipe does not support single op compilation!'}, 'In op [fix_pipe], [fix_pipe does not support single op compilation!]')
], optype: [FixPipe])
Compile op[/model/neck/pan_blocks.1/m/m.2/conv2/conv/Convfunction_graph_54//model/neck/pan_blocks.1/m/m.2/conv2/conv/Conv_pass.1,[/model/neck/pan_blocks.1/m/m.2/Add,/model/neck/pan_blocks.1/m/m.2/Add]] failed, oppath[/usr/local/Ascend/ascend-toolkit/8.0.RC2.alpha001/opp/built-in/op_impl/ai_core/tbe/impl/fix_pipe.py], optype[FixPipe], taskID[2442]. Please check op's compilation error message.[FUNC:ReportBuildErrMessage][FILE:fusion_manager.cc][LINE:748]
Failed to compile Op [/model/neck/fpn_blocks.1/m/m.2/conv2/conv/Convfunction_graph_48//model/neck/fpn_blocks.1/m/m.2/conv2/conv/Conv_pass.1,[/model/neck/fpn_blocks.1/m/m.2/Add,/model/neck/fpn_blocks.1/m/m.2/Add]]. (oppath: [Compile /usr/local/Ascend/ascend-toolkit/8.0.RC2.alpha001/opp/built-in/op_impl/ai_core/tbe/impl/fix_pipe.py failed with errormsg/stack: File "/usr/local/Ascend/ascend-toolkit/latest/python/site-packages/tbe/common/utils/errormgr/error_manager_vector.py", line 284, in raise_err_specific_reson
raise RuntimeError(args_dict, msg)
RuntimeError: ({'errCode': 'E61001', 'op_name': 'fix_pipe', 'reason': 'fix_pipe does not support single op compilation!'}, 'In op [fix_pipe], [fix_pipe does not support single op compilation!]')
], optype: [FixPipe])
Compile op[/model/neck/fpn_blocks.1/m/m.2/conv2/conv/Convfunction_graph_48//model/neck/fpn_blocks.1/m/m.2/conv2/conv/Conv_pass.1,[/model/neck/fpn_blocks.1/m/m.2/Add,/model/neck/fpn_blocks.1/m/m.2/Add]] failed, oppath[/usr/local/Ascend/ascend-toolkit/8.0.RC2.alpha001/opp/built-in/op_impl/ai_core/tbe/impl/fix_pipe.py], optype[FixPipe], taskID[2454]. Please check op's compilation error message.[FUNC:ReportBuildErrMessage][FILE:fusion_manager.cc][LINE:748]
[SubGraphOpt][Compile][ProcFailedCompTask] Thread[281469742205616] recompile single op[/model/neck/fpn_blocks.1/m/m.0/conv2/conv/Convfunction_graph_46//model/neck/fpn_blocks.1/m/m.0/conv2/conv/Conv_pass.1] failed[FUNC:ProcessAllFailedCompileTasks][FILE:tbe_op_store_adapter.cc][LINE:962]
[SubGraphOpt][Compile][ProcFailedCompTask] Thread[281469742205616] recompile single op[/model/neck/fpn_blocks.1/m/m.1/conv2/conv/Convfunction_graph_47//model/neck/fpn_blocks.1/m/m.1/conv2/conv/Conv_pass.1] failed[FUNC:ProcessAllFailedCompileTasks][FILE:tbe_op_store_adapter.cc][LINE:962]
[SubGraphOpt][Compile][ProcFailedCompTask] Thread[281469742205616] recompile single op[/model/neck/fpn_blocks.1/m/m.2/conv2/conv/Convfunction_graph_48//model/neck/fpn_blocks.1/m/m.2/conv2/conv/Conv_pass.1] failed[FUNC:ProcessAllFailedCompileTasks][FILE:tbe_op_store_adapter.cc][LINE:962]
[SubGraphOpt][Compile][ProcFailedCompTask] Thread[281469742205616] recompile single op[/model/neck/pan_blocks.1/m/m.0/conv2/conv/Convfunction_graph_52//model/neck/pan_blocks.1/m/m.0/conv2/conv/Conv_pass.1] failed[FUNC:ProcessAllFailedCompileTasks][FILE:tbe_op_store_adapter.cc][LINE:962]
[SubGraphOpt][Compile][ProcFailedCompTask] Thread[281469742205616] recompile single op[/model/neck/pan_blocks.1/m/m.1/conv2/conv/Convfunction_graph_53//model/neck/pan_blocks.1/m/m.1/conv2/conv/Conv_pass.1] failed[FUNC:ProcessAllFailedCompileTasks][FILE:tbe_op_store_adapter.cc][LINE:962]
[SubGraphOpt][Compile][ProcFailedCompTask] Thread[281469742205616] recompile single op[/model/neck/pan_blocks.1/m/m.2/conv2/conv/Convfunction_graph_54//model/neck/pan_blocks.1/m/m.2/conv2/conv/Conv_pass.1] failed[FUNC:ProcessAllFailedCompileTasks][FILE:tbe_op_store_adapter.cc][LINE:962]
[SubGraphOpt][Compile][ProcFailedCompTask] Thread[281469742205616] recompile single op[/model/neck/pan_blocks.0/m/m.0/conv2/conv/Convfunction_graph_49//model/neck/pan_blocks.0/m/m.0/conv2/conv/Conv_pass.1] failed[FUNC:ProcessAllFailedCompileTasks][FILE:tbe_op_store_adapter.cc][LINE:962]
[SubGraphOpt][Compile][ProcFailedCompTask] Thread[281469742205616] recompile single op[/model/neck/pan_blocks.0/m/m.1/conv2/conv/Convfunction_graph_50//model/neck/pan_blocks.0/m/m.1/conv2/conv/Conv_pass.1] failed[FUNC:ProcessAllFailedCompileTasks][FILE:tbe_op_store_adapter.cc][LINE:962]
[SubGraphOpt][Compile][ProcFailedCompTask] Thread[281469742205616] recompile single op[/model/neck/pan_blocks.0/m/m.2/conv2/conv/Convfunction_graph_51//model/neck/pan_blocks.0/m/m.2/conv2/conv/Conv_pass.1] failed[FUNC:ProcessAllFailedCompileTasks][FILE:tbe_op_store_adapter.cc][LINE:962]
[SubGraphOpt][Compile][ParalCompOp] Thread[281469742205616] process fail task failed[FUNC:ParallelCompileOp][FILE:tbe_op_store_adapter.cc][LINE:1010]
Call OptimizeFusedGraph failed, ret:-1, engine_name:AIcoreEngine, graph_name:partition0_rank122_new_sub_graph943[FUNC:OptimizeSubGraph][FILE:graph_optimize.cc][LINE:126]
build graph failed, graph id:0, ret:-1[FUNC:BuildModelWithGraphId][FILE:ge_generator.cc][LINE:1615]
GenerateOfflineModel execute failed.
点赞
收藏
回复
分享
举报
已解决
如果换成 atc --model=/app/resources/bs1_simplify.onnx --framework=5 --output=/app/resources/model_fp16 --soc_version=Ascend310B1 --log=debug --precision_mode=force_fp16，都会失败， 在300V 和910B 上都进行了尝试 都不行应该不是 设备资源不够的问题吧
报错
ATC run failed, Please check the detail log, Try 'atc --help' for more information
E40021: 2025-01-18-01:43:35.711.466 Failed to compile Op [/model/neck/fpn_blocks.0/m/m.2/conv2/conv/Convfunction_graph_45//model/neck/fpn_blocks.0/m/m.2/conv2/conv/Conv_pass.1,[/model/neck/fpn_blocks.0/m/m.2/Add,/model/neck/fpn_blocks.0/m/m.2/Add]]. (oppath: [Compile /usr/local/Ascend/ascend-toolkit/8.0.RC2.alpha001/opp/built-in/op_impl/ai_core/tbe/impl/fix_pipe.py failed with errormsg/stack: File "/usr/local/Ascend/ascend-toolkit/latest/python/site-packages/tbe/common/utils/errormgr/error_manager_vector.py", line 284, in raise_err_specific_reson
raise RuntimeError(args_dict, msg)
RuntimeError: ({'errCode': 'E61001', 'op_name': 'fix_pipe', 'reason': 'fix_pipe does not support single op compilation!'}, 'In op [fix_pipe], [fix_pipe does not support single op compilation!]')
], optype: [FixPipe])
Solution: See the host log for details, and then check the Python stack where the error log is reported.
TraceBack (most recent call last):
Compile op[/model/neck/fpn_blocks.0/m/m.2/conv2/conv/Convfunction_graph_45//model/neck/fpn_blocks.0/m/m.2/conv2/conv/Conv_pass.1,[/model/neck/fpn_blocks.0/m/m.2/Add,/model/neck/fpn_blocks.0/m/m.2/Add]] failed, oppath[/usr/local/Ascend/ascend-toolkit/8.0.RC2.alpha001/opp/built-in/op_impl/ai_core/tbe/impl/fix_pipe.py], optype[FixPipe], taskID[2364]. Please check op's compilation error message.[FUNC:ReportBuildErrMessage][FILE:fusion_manager.cc][LINE:748]
[SubGraphOpt][Compile][ProcFailedCompTask] Thread[281470438439600] recompile single op[/model/neck/fpn_blocks.0/m/m.0/conv2/conv/Convfunction_graph_43//model/neck/fpn_blocks.0/m/m.0/conv2/conv/Conv_pass.1] failed[FUNC:ProcessAllFailedCompileTasks][FILE:tbe_op_store_adapter.cc][LINE:962]
[SubGraphOpt][Compile][ProcFailedCompTask] Thread[281470438439600] recompile single op[/model/neck/fpn_blocks.0/m/m.1/conv2/conv/Convfunction_graph_44//model/neck/fpn_blocks.0/m/m.1/conv2/conv/Conv_pass.1] failed[FUNC:ProcessAllFailedCompileTasks][FILE:tbe_op_store_adapter.cc][LINE:962]
[SubGraphOpt][Compile][ProcFailedCompTask] Thread[281470438439600] recompile single op[/model/neck/fpn_blocks.0/m/m.2/conv2/conv/Convfunction_graph_45//model/neck/fpn_blocks.0/m/m.2/conv2/conv/Conv_pass.1] failed[FUNC:ProcessAllFailedCompileTasks][FILE:tbe_op_store_adapter.cc][LINE:962]
[SubGraphOpt][Compile][ParalCompOp] Thread[281470438439600] process fail task failed[FUNC:ParallelCompileOp][FILE:tbe_op_store_adapter.cc][LINE:1010]
[SubGraphOpt][Compile][CompOpOnly] CompileOp failed.[FUNC:CompileOpOnly][FILE:op_compiler.cc][LINE:1119]
[GraphOpt][FusedGraph][RunCompile] Failed to compile graph with compiler Normal mode Op Compiler[FUNC:SubGraphCompile][FILE:fe_graph_optimizer.cc][LINE:1385]
Call OptimizeFusedGraph failed, ret:-1, engine_name:AIcoreEngine, graph_name:partition0_rank120_new_sub_graph941[FUNC:OptimizeSubGraph][FILE:graph_optimize.cc][LINE:126]
subgraph 119 optimize failed[FUNC:OptimizeSubGraphWithMultiThreads][FILE:graph_manager.cc][LINE:1024]
Failed to compile Op [/model/neck/pan_blocks.1/m/m.2/conv2/conv/Convfunction_graph_54//model/neck/pan_blocks.1/m/m.2/conv2/conv/Conv_pass.1,[/model/neck/pan_blocks.1/m/m.2/Add,/model/neck/pan_blocks.1/m/m.2/Add]]. (oppath: [Compile /usr/local/Ascend/ascend-toolkit/8.0.RC2.alpha001/opp/built-in/op_impl/ai_core/tbe/impl/fix_pipe.py failed with errormsg/stack: File "/usr/local/Ascend/ascend-toolkit/latest/python/site-packages/tbe/common/utils/errormgr/error_manager_vector.py", line 284, in raise_err_specific_reson
raise RuntimeError(args_dict, msg)
RuntimeError: ({'errCode': 'E61001', 'op_name': 'fix_pipe', 'reason': 'fix_pipe does not support single op compilation!'}, 'In op [fix_pipe], [fix_pipe does not support single op compilation!]')
], optype: [FixPipe])
Compile op[/model/neck/pan_blocks.1/m/m.2/conv2/conv/Convfunction_graph_54//model/neck/pan_blocks.1/m/m.2/conv2/conv/Conv_pass.1,[/model/neck/pan_blocks.1/m/m.2/Add,/model/neck/pan_blocks.1/m/m.2/Add]] failed, oppath[/usr/local/Ascend/ascend-toolkit/8.0.RC2.alpha001/opp/built-in/op_impl/ai_core/tbe/impl/fix_pipe.py], optype[FixPipe], taskID[2442]. Please check op's compilation error message.[FUNC:ReportBuildErrMessage][FILE:fusion_manager.cc][LINE:748]
Failed to compile Op [/model/neck/fpn_blocks.1/m/m.2/conv2/conv/Convfunction_graph_48//model/neck/fpn_blocks.1/m/m.2/conv2/conv/Conv_pass.1,[/model/neck/fpn_blocks.1/m/m.2/Add,/model/neck/fpn_blocks.1/m/m.2/Add]]. (oppath: [Compile /usr/local/Ascend/ascend-toolkit/8.0.RC2.alpha001/opp/built-in/op_impl/ai_core/tbe/impl/fix_pipe.py failed with errormsg/stack: File "/usr/local/Ascend/ascend-toolkit/latest/python/site-packages/tbe/common/utils/errormgr/error_manager_vector.py", line 284, in raise_err_specific_reson
raise RuntimeError(args_dict, msg)
RuntimeError: ({'errCode': 'E61001', 'op_name': 'fix_pipe', 'reason': 'fix_pipe does not support single op compilation!'}, 'In op [fix_pipe], [fix_pipe does not support single op compilation!]')
], optype: [FixPipe])
Compile op[/model/neck/fpn_blocks.1/m/m.2/conv2/conv/Convfunction_graph_48//model/neck/fpn_blocks.1/m/m.2/conv2/conv/Conv_pass.1,[/model/neck/fpn_blocks.1/m/m.2/Add,/model/neck/fpn_blocks.1/m/m.2/Add]] failed, oppath[/usr/local/Ascend/ascend-toolkit/8.0.RC2.alpha001/opp/built-in/op_impl/ai_core/tbe/impl/fix_pipe.py], optype[FixPipe], taskID[2454]. Please check op's compilation error message.[FUNC:ReportBuildErrMessage][FILE:fusion_manager.cc][LINE:748]
[SubGraphOpt][Compile][ProcFailedCompTask] Thread[281469742205616] recompile single op[/model/neck/fpn_blocks.1/m/m.0/conv2/conv/Convfunction_graph_46//model/neck/fpn_blocks.1/m/m.0/conv2/conv/Conv_pass.1] failed[FUNC:ProcessAllFailedCompileTasks][FILE:tbe_op_store_adapter.cc][LINE:962]
[SubGraphOpt][Compile][ProcFailedCompTask] Thread[281469742205616] recompile single op[/model/neck/fpn_blocks.1/m/m.1/conv2/conv/Convfunction_graph_47//model/neck/fpn_blocks.1/m/m.1/conv2/conv/Conv_pass.1] failed[FUNC:ProcessAllFailedCompileTasks][FILE:tbe_op_store_adapter.cc][LINE:962]
[SubGraphOpt][Compile][ProcFailedCompTask] Thread[281469742205616] recompile single op[/model/neck/fpn_blocks.1/m/m.2/conv2/conv/Convfunction_graph_48//model/neck/fpn_blocks.1/m/m.2/conv2/conv/Conv_pass.1] failed[FUNC:ProcessAllFailedCompileTasks][FILE:tbe_op_store_adapter.cc][LINE:962]
[SubGraphOpt][Compile][ProcFailedCompTask] Thread[281469742205616] recompile single op[/model/neck/pan_blocks.1/m/m.0/conv2/conv/Convfunction_graph_52//model/neck/pan_blocks.1/m/m.0/conv2/conv/Conv_pass.1] failed[FUNC:ProcessAllFailedCompileTasks][FILE:tbe_op_store_adapter.cc][LINE:962]
[SubGraphOpt][Compile][ProcFailedCompTask] Thread[281469742205616] recompile single op[/model/neck/pan_blocks.1/m/m.1/conv2/conv/Convfunction_graph_53//model/neck/pan_blocks.1/m/m.1/conv2/conv/Conv_pass.1] failed[FUNC:ProcessAllFailedCompileTasks][FILE:tbe_op_store_adapter.cc][LINE:962]
[SubGraphOpt][Compile][ProcFailedCompTask] Thread[281469742205616] recompile single op[/model/neck/pan_blocks.1/m/m.2/conv2/conv/Convfunction_graph_54//model/neck/pan_blocks.1/m/m.2/conv2/conv/Conv_pass.1] failed[FUNC:ProcessAllFailedCompileTasks][FILE:tbe_op_store_adapter.cc][LINE:962]
[SubGraphOpt][Compile][ProcFailedCompTask] Thread[281469742205616] recompile single op[/model/neck/pan_blocks.0/m/m.0/conv2/conv/Convfunction_graph_49//model/neck/pan_blocks.0/m/m.0/conv2/conv/Conv_pass.1] failed[FUNC:ProcessAllFailedCompileTasks][FILE:tbe_op_store_adapter.cc][LINE:962]
[SubGraphOpt][Compile][ProcFailedCompTask] Thread[281469742205616] recompile single op[/model/neck/pan_blocks.0/m/m.1/conv2/conv/Convfunction_graph_50//model/neck/pan_blocks.0/m/m.1/conv2/conv/Conv_pass.1] failed[FUNC:ProcessAllFailedCompileTasks][FILE:tbe_op_store_adapter.cc][LINE:962]
[SubGraphOpt][Compile][ProcFailedCompTask] Thread[281469742205616] recompile single op[/model/neck/pan_blocks.0/m/m.2/conv2/conv/Convfunction_graph_51//model/neck/pan_blocks.0/m/m.2/conv2/conv/Conv_pass.1] failed[FUNC:ProcessAllFailedCompileTasks][FILE:tbe_op_store_adapter.cc][LINE:962]
[SubGraphOpt][Compile][ParalCompOp] Thread[281469742205616] process fail task failed[FUNC:ParallelCompileOp][FILE:tbe_op_store_adapter.cc][LINE:1010]
Call OptimizeFusedGraph failed, ret:-1, engine_name:AIcoreEngine, graph_name:partition0_rank122_new_sub_graph943[FUNC:OptimizeSubGraph][FILE:graph_optimize.cc][LINE:126]
build graph failed, graph id:0, ret:-1[FUNC:BuildModelWithGraphId][FILE:ge_generator.cc][LINE:1615]
GenerateOfflineModel execute failed.