ATC模型转换时报错
收藏回复举报
ATC模型转换时报错
t('forum.solved') 已解决
发表于2024-08-31 09:54:47
0 查看
我的固件版本是1.0.13。CANN为6.0 RC1,系统为ubuntu18.04。在使用atc  --input_shape=input:1,3,640,640 --input_format=NCHW  --soc_version=Ascend310 --framework=5 --model=DE.onnx  --output=DE --precision_mode=allow_fp32_to_fp16命令时报错。报错内容为E49999: Inner Error! 

E49999  Compile op[/encoder_level1/mhca_blks.1/MHCA_layers.0/layers.0.0/norm/Expand,[_v_895_102,_v_895_102,/encoder_level1/mhca_blks.1/MHCA_layers.0/layers.0.0/norm/Expand]] failed, oppath[/usr/local/Ascend/ascend-toolkit/6.0.RC1/opp/op_impl/built-in/ai_core/tbe/impl/expand_d.py], optype[ExpandD], taskID[603]. Please check op's compilation error message.[FUNC:ReportBuildErrMessage][FILE:fusion_op.cc][LINE:856] 

        Compile op[/encoder_level1/mhca_blks.1/MHCA_layers.1/layers.0.1/layers.0.1.0/Expand,[_v_897_103,_v_897_103,/encoder_level1/mhca_blks.1/MHCA_layers.1/layers.0.1/layers.0.1.0/Expand]] failed, oppath[/usr/local/Ascend/ascend-toolkit/6.0.RC1/opp/op_impl/built-in/ai_core/tbe/impl/expand_d.py], optype[ExpandD], taskID[673]. Please check op's compilation error message.[FUNC:ReportBuildErrMessage][FILE:fusion_op.cc][LINE:856] 

        [SubGraphOpt][Compile][ProcFailedCompTask] Thread[281469205337824] recompile single op[/encoder_level1/mhca_blks.1/MHCA_layers.1/final_norm/Expand] failed[FUNC:ProcessAllFailedCompileTasks][FILE:tbe_op_store_adapter.cc][LINE:827] 

        [SubGraphOpt][Compile][ProcFailedCompTask] Thread[281469205337824] recompile single op[/encoder_level1/mhca_blks.1/MHCA_layers.1/layers.0.1/layers.0.1.0/Expand] failed[FUNC:ProcessAllFailedCompileTasks][FILE:tbe_op_store_adapter.cc][LINE:827] 

        [SubGraphOpt][Compile][ParalCompOp] Thread[281469205337824] process fail task failed[FUNC:ParallelCompileOp][FILE:tbe_op_store_adapter.cc][LINE:867] 

        [SubGraphOpt][Compile][CompOpOnly] CompileOp failed.[FUNC:CompileOpOnly][FILE:op_compiler.cc][LINE:994] 

        [GraphOpt][FusedGraph][RunCompile] Failed to compile graph with compiler Normal mode Op Compiler[FUNC:SubGraphCompile][FILE:fe_graph_optimizer.cc][LINE:1467] 

        Call OptimizeFusedGraph failed, ret:-1, engine_name:AIcoreEngine, graph_name:partition0_rank30_new_sub_graph268[FUNC:OptimizeSubGraph][FILE:graph_optimize.cc][LINE:130] 

        [SubGraphOpt][Compile][ProcFailedCompTask] Thread[281469485369056] recompile single op[/encoder_level1/mhca_blks.1/MHCA_layers.1/layers.0.0/norm/Expand] failed[FUNC:ProcessAllFailedCompileTasks][FILE:tbe_op_store_adapter.cc][LINE:827] 

        [SubGraphOpt][Compile][ProcFailedCompTask] Thread[281469485369056] recompile single op[/encoder_level1/mhca_blks.1/MHCA_layers.0/layers.0.1/layers.0.1.0/Expand] failed[FUNC:ProcessAllFailedCompileTasks][FILE:tbe_op_store_adapter.cc][LINE:827] 

        [SubGraphOpt][Compile][ProcFailedCompTask] Thread[281469485369056] recompile single op[/encoder_level1/mhca_blks.1/MHCA_layers.0/final_norm/Expand] failed[FUNC:ProcessAllFailedCompileTasks][FILE:tbe_op_store_adapter.cc][LINE:827] 

        [SubGraphOpt][Compile][ParalCompOp] Thread[281469485369056] process fail task failed[FUNC:ParallelCompileOp][FILE:tbe_op_store_adapter.cc][LINE:867] 

        [SubGraphOpt][Compile][ProcFailedCompTask] Thread[281468794291936] recompile single op[/encoder_level1/mhca_blks.0/MHCA_layers.1/final_norm/Expand] failed[FUNC:ProcessAllFailedCompileTasks][FILE:tbe_op_store_adapter.cc][LINE:827] 

        [SubGraphOpt][Compile][ProcFailedCompTask] Thread[281468794291936] recompile single op[/encoder_level1/mhca_blks.0/MHCA_layers.1/layers.0.1/layers.0.1.0/Expand] failed[FUNC:ProcessAllFailedCompileTasks][FILE:tbe_op_store_adapter.cc][LINE:827] 

        [SubGraphOpt][Compile][ParalCompOp] Thread[281468794291936] process fail task failed[FUNC:ParallelCompileOp][FILE:tbe_op_store_adapter.cc][LINE:867] 

        Call OptimizeFusedGraph failed, ret:-1, engine_name:AIcoreEngine, graph_name:partition0_rank25_new_sub_graph262[FUNC:OptimizeSubGraph][FILE:graph_optimize.cc][LINE:130] 

        Call OptimizeFusedGraph failed, ret:-1, engine_name:AIcoreEngine, graph_name:partition0_rank19_new_sub_graph255[FUNC:OptimizeSubGraph][FILE:graph_optimize.cc][LINE:130] 

        [SubGraphOpt][Compile][ProcFailedCompTask] Thread[281469180159712] recompile single op[/encoder_level1/mhca_blks.0/MHCA_layers.1/layers.0.0/norm/Expand] failed[FUNC:ProcessAllFailedCompileTasks][FILE:tbe_op_store_adapter.cc][LINE:827] 

        [SubGraphOpt][Compile][ProcFailedCompTask] Thread[281469180159712] recompile single op[/encoder_level1/mhca_blks.0/MHCA_layers.0/final_norm/Expand] failed[FUNC:ProcessAllFailedCompileTasks][FILE:tbe_op_store_adapter.cc][LINE:827] 

        [SubGraphOpt][Compile][ProcFailedCompTask] Thread[281469180159712] recompile single op[/encoder_level1/mhca_blks.0/MHCA_layers.0/layers.0.1/layers.0.1.0/Expand] failed[FUNC:ProcessAllFailedCompileTasks][FILE:tbe_op_store_adapter.cc][LINE:827] 

        [SubGraphOpt][Compile][ParalCompOp] Thread[281469180159712] process fail task failed[FUNC:ParallelCompileOp][FILE:tbe_op_store_adapter.cc][LINE:867] 

        Call OptimizeFusedGraph failed, ret:-1, engine_name:AIcoreEngine, graph_name:partition0_rank13_new_sub_graph248[FUNC:OptimizeSubGraph][FILE:graph_optimize.cc][LINE:130] 

        [SubGraphOpt][Compile][ProcFailedCompTask] Thread[281468802684640] recompile single op[/decoder_level1/mhca_blks.0/MHCA_layers.1/layers.0.0/norm/Expand] failed[FUNC:ProcessAllFailedCompileTasks][FILE:tbe_op_store_adapter.cc][LINE:827] 

        [SubGraphOpt][Compile][ProcFailedCompTask] Thread[281468802684640] recompile single op[/decoder_level1/mhca_blks.0/MHCA_layers.0/layers.0.1/layers.0.1.0/Expand] failed[FUNC:ProcessAllFailedCompileTasks][FILE:tbe_op_store_adapter.cc][LINE:827] 

        [SubGraphOpt][Compile][ProcFailedCompTask] Thread[281468802684640] recompile single op[/decoder_level1/mhca_blks.0/MHCA_layers.0/final_norm/Expand] failed[FUNC:ProcessAllFailedCompileTasks][FILE:tbe_op_store_adapter.cc][LINE:827] 

        [SubGraphOpt][Compile][ParalCompOp] Thread[281468802684640] process fail task failed[FUNC:ParallelCompileOp][FILE:tbe_op_store_adapter.cc][LINE:867] 

        Call OptimizeFusedGraph failed, ret:-1, engine_name:AIcoreEngine, graph_name:partition0_rank49_new_sub_graph288[FUNC:OptimizeSubGraph][FILE:graph_optimize.cc][LINE:130] 

        [SubGraphOpt][Compile][ProcFailedCompTask] Thread[281468802684640] recompile single op[/decoder_level1/mhca_blks.1/MHCA_layers.1/final_norm/Expand] failed[FUNC:ProcessAllFailedCompileTasks][FILE:tbe_op_store_adapter.cc][LINE:827] 

        [SubGraphOpt][Compile][ProcFailedCompTask] Thread[281468802684640] recompile single op[/decoder_level1/mhca_blks.1/MHCA_layers.1/layers.0.1/layers.0.1.0/Expand] failed[FUNC:ProcessAllFailedCompileTasks][FILE:tbe_op_store_adapter.cc][LINE:827] 

        Call OptimizeFusedGraph failed, ret:-1, engine_name:AIcoreEngine, graph_name:partition0_rank66_new_sub_graph308[FUNC:OptimizeSubGraph][FILE:graph_optimize.cc][LINE:130] 

        [SubGraphOpt][Compile][ProcFailedCompTask] Thread[281469485369056] recompile single op[/decoder_level1/mhca_blks.0/MHCA_layers.1/final_norm/Expand] failed[FUNC:ProcessAllFailedCompileTasks][FILE:tbe_op_store_adapter.cc][LINE:827] 

        [SubGraphOpt][Compile][ProcFailedCompTask] Thread[281469485369056] recompile single op[/decoder_level1/mhca_blks.0/MHCA_layers.1/layers.0.1/layers.0.1.0/Expand] failed[FUNC:ProcessAllFailedCompileTasks][FILE:tbe_op_store_adapter.cc][LINE:827] 

        Call OptimizeFusedGraph failed, ret:-1, engine_name:AIcoreEngine, graph_name:partition0_rank61_new_sub_graph302[FUNC:OptimizeSubGraph][FILE:graph_optimize.cc][LINE:130] 

        [SubGraphOpt][Compile][ProcFailedCompTask] Thread[281469205337824] recompile single op[/decoder_level1/mhca_blks.1/MHCA_layers.1/layers.0.0/norm/Expand] failed[FUNC:ProcessAllFailedCompileTasks][FILE:tbe_op_store_adapter.cc][LINE:827] 

        [SubGraphOpt][Compile][ProcFailedCompTask] Thread[281469205337824] recompile single op[/decoder_level1/mhca_blks.1/MHCA_layers.0/layers.0.1/layers.0.1.0/Expand] failed[FUNC:ProcessAllFailedCompileTasks][FILE:tbe_op_store_adapter.cc][LINE:827] 

        [SubGraphOpt][Compile][ProcFailedCompTask] Thread[281469205337824] recompile single op[/decoder_level1/mhca_blks.1/MHCA_layers.0/final_norm/Expand] failed[FUNC:ProcessAllFailedCompileTasks][FILE:tbe_op_store_adapter.cc][LINE:827] 

        Call OptimizeFusedGraph failed, ret:-1, engine_name:AIcoreEngine, graph_name:partition0_rank55_new_sub_graph295[FUNC:OptimizeSubGraph][FILE:graph_optimize.cc][LINE:130] 

        [SubGraphOpt][Compile][ProcFailedCompTask] Thread[281469188552416] recompile single op[/decoder_level1/mhca_blks.0/MHCA_layers.0/layers.0.0/norm/Expand] failed[FUNC:ProcessAllFailedCompileTasks][FILE:tbe_op_store_adapter.cc][LINE:827] 

        [SubGraphOpt][Compile][ParalCompOp] Thread[281469188552416] process fail task failed[FUNC:ParallelCompileOp][FILE:tbe_op_store_adapter.cc][LINE:867] 

        Call OptimizeFusedGraph failed, ret:-1, engine_name:AIcoreEngine, graph_name:partition0_rank39_new_sub_graph277[FUNC:OptimizeSubGraph][FILE:graph_optimize.cc][LINE:130] 

        [SubGraphOpt][Compile][ProcFailedCompTask] Thread[281468760721120] recompile single op[/encoder_level1/mhca_blks.0/MHCA_layers.0/layers.0.0/norm/Expand] failed[FUNC:ProcessAllFailedCompileTasks][FILE:tbe_op_store_adapter.cc][LINE:827] 

        [SubGraphOpt][Compile][ParalCompOp] Thread[281468760721120] process fail task failed[FUNC:ParallelCompileOp][FILE:tbe_op_store_adapter.cc][LINE:867] 

        Call OptimizeFusedGraph failed, ret:-1, engine_name:AIcoreEngine, graph_name:partition0_rank8_new_sub_graph240[FUNC:OptimizeSubGraph][FILE:graph_optimize.cc][LINE:130] 

        [SubGraphOpt][Compile][ProcFailedCompTask] Thread[281469154981600] recompile single op[/decoder_level1/mhca_blks.1/MHCA_layers.0/layers.0.0/norm/Expand] failed[FUNC:ProcessAllFailedCompileTasks][FILE:tbe_op_store_adapter.cc][LINE:827] 

        [SubGraphOpt][Compile][ParalCompOp] Thread[281469154981600] process fail task failed[FUNC:ParallelCompileOp][FILE:tbe_op_store_adapter.cc][LINE:867] 

        Call OptimizeFusedGraph failed, ret:-1, engine_name:AIcoreEngine, graph_name:partition0_rank44_new_sub_graph282[FUNC:OptimizeSubGraph][FILE:graph_optimize.cc][LINE:130] 

        [SubGraphOpt][Compile][ProcFailedCompTask] Thread[281469196945120] recompile single op[/encoder_level1/mhca_blks.1/MHCA_layers.0/layers.0.0/norm/Expand] failed[FUNC:ProcessAllFailedCompileTasks][FILE:tbe_op_store_adapter.cc][LINE:827] 

        [SubGraphOpt][Compile][ParalCompOp] Thread[281469196945120] process fail task failed[FUNC:ParallelCompileOp][FILE:tbe_op_store_adapter.cc][LINE:867] 

        Call OptimizeFusedGraph failed, ret:-1, engine_name:AIcoreEngine, graph_name:partition0_rank4_new_sub_graph236[FUNC:OptimizeSubGraph][FILE:graph_optimize.cc][LINE:130] 

        subgraph 3 optimize failed[FUNC:OptimizeSubGraphWithMultiThreads][FILE:graph_manager.cc][LINE:714] 

        build graph failed, graph id:0, ret:-1[FUNC:BuildModel][FILE:ge_generator.cc][LINE:1496] 

        Solution: Please contact support engineer.无log。定位不到问题。https://www.123pan.com/s/hM8mjv-70gWv.html提取码:AevI这是我的onnx文件。

我要发帖子