DUMP_GRAPH_LEVEL
Description
DUMP_GRAPH_LEVEL sets the graph to dump.
This environment variable can be configured in either of the following ways. Both of them are used to control the number of flushed graphs. You can use them as required. Note that the two configuration methods cannot be used together.
- Valid values:
- 1: dumps graphs in all phases.
- 2: dumps graphs in the trustlist phase. For details about the trustlisted graphs, see the Trustlisted or Not column in Table 1.
- 3: dumps the last generated graph, that is, the graph optimized and compiled by the GE.
- 4: dumps the earliest generated graph, that is, the build entry graph provided to the software stack after the GE parses and maps operators. This graph structure has not been compiled or optimized by the GE.
- Set it to a string separated by vertical bars (|):
For example, if this environment variable is set to aa|bb, graphs whose names contain aa and bb are dumped. aa and bb must be set to valid character strings during graph build. Valid strings can be obtained from a full dump of the graphs.
DUMP_GRAPH_LEVEL takes effect only when DUMP_GE_GRAPH is enabled. The default value is 2.
Example
- Set this environment variable to a value:
export DUMP_GRAPH_LEVEL=1
- Set this environment variable to a character string separated by vertical bars (|):
export DUMP_GRAPH_LEVEL="PreRunBegin|AfterInfershape"
Restrictions
- If this environment variable is set to an invalid value, undefined behavior may occur.
- If the operator dump data collection function is enabled, see ge.exec.enableDump in Graph Development. Even if the DUMP_GRAPH_LEVEL environment variable is not configured or export DUMP_GRAPH_LEVEL="PreRunBegin|AfterInfershape" is configured but does not contain the Build string, the subgraph ge_proto_xxxx_Build.txt is dumped.
- This environment variable must be used together with DUMP_GE_GRAPH. That is, when DUMP_GE_GRAPH is enabled, DUMP_GRAPH_LEVEL can be used to control the generated dump graphs. For details about the full dump graph information, see Table 1.
Table 1 Description of dump graph details No.
Subgraph
Component
Phase
Trustlisted or Not
1
ge_proto_xxxx_GraphPreRunBegin.txt
GE
Graph before build.
-
2
ge_proto_xxxx_AfterFlowGraphPartition.txt
GE
Graph after flow partitioning (flow partitioning is a partitioning mode used in DataFlow).
-
3
ge_proto_xxxx_AfterParallelPartitioner.txt
GE
Graph after pipeline parallel partitioning (pipeline here refers to PP in the backend inference scenario).
-
4
ge_proto_xxxx_PreRunBegin.txt
GE
Original graph structure.
Trustlisted
5
ge_proto_xxxx_RunCustomPassBeforeInfershape.txt
GE
Exit graph of the custom pass before InferShape.
Trustlisted
6
ge_proto_xxxx_PreRunAfterInitPreparation.txt
FE
Graph structure after all initialization processing in the graph preparation phase.
-
7
ge_proto_xxxx_PreRunAfterHandleSummaryOp.txt
GE
Graph structure after Summary node processing.
-
8
ge_proto_xxxx_PrepareAfterCheckAndUpdateInput.txt
GE
Graph structure after the graph input is checked and updated.
-
9
ge_proto_xxxx_PrepareAfterGraphEquivalentTransformation.txt
GE
Graph structure after equivalent replacement of For-loop graph structures with While-loop graph structures.
-
10
ge_proto_xxxx_PrepareAfterProcessOutput.txt
GE
Graph structure after graph data processing.
-
11
ge_proto_xxxx_PrepareAfterOptimizeAfterGraphNormalization.txt
GE
Exit graph after graph optimization following graph normalization.
-
12
ge_proto_xxxx_PrepareAfterInsertAipp.txt
GE
Graph structure after AIPP processing (when AIPP parameters are configured).
-
13
ge_proto_xxxx_PrepareAfterProcessAippNodesDataFormat.txt
GE
Exit graph after AIPP node format update.
-
14
ge_proto_xxxx_PreRunAfterNormalizeGraph.txt
GE
Exit graph after graph normalization.
Trustlisted
15
ge_proto_xxxx_PreRunAfterOptimizeGraphInit.txt
GE
Exit graph after graph optimization and initialization.
-
16
ge_proto_xxxx_OptimizeGraph_TagNoConstFoldingAfter.txt
FE
Used in quantization scenarios. The FE tags operators that should skip constant folding. During constant folding, the GE checks this tag and skips constant folding if the tag exists.
-
17
ge_proto_xxxx_HcclAfterOptimizeGraphPrepare.txt
HCCL
Graph after optimization in the HCCL graph preparation phase.
-
18
ge_proto_xxxx_PreRunAfterOptimizeGraphPrepare.txt
GE
Graph structure after original-graph preparation via each operator information library (by calling the OptimizeGraphPrepare API).
-
19
ge_proto_xxxx_PrepareAfterProcessBeforeInfershape.txt
GE
Graph structure after dead-edge elimination for conditional operators.
-
20
ge_proto_xxxx_AfterFirstInferformat.txt
GE
Graph structure after full-graph format inference.
-
21
ge_proto_xxxx_AfterInfershape.txt
GE
Graph structure after full-graph shape inference, accompanied by constant folding.
Trustlisted
22
ge_proto_xxxx_PrepareAfterInferFormatAndShape.txt
GE
Graph structure after all InferFormat and InferShape processing. A second full-graph InferFormat occurs between this and the previous graph.
-
23
ge_proto_xxxx_RunCustomPass_AfterInferShape.txt
GE
Exit graph of the custom pass after InferShape.
Trustlisted
24
ge_proto_xxxx_AfterSecondInferformat.txt
GE
Graph after the second format inference.
-
25
ge_proto_xxxx_PrepareAfterCtrlFlowPreProcess.txt
GE
Graph structure after conditional operator preprocessing.
-
26
ge_proto_xxxx_PrepareAfterGetDynamicOutputShape.txt
GE
Graph structure after graph output processing in dynamic-profile scenarios.
-
27
ge_proto_xxxx_PrepareAfterProcessAippStage2.txt
GE
Graph structure after graph input node processing in AIPP mode.
-
28
ge_proto_xxxx_PrepareAfterPrepareOptimize.txt
GE
Graph structure after optimization in the graph preparation phase.
-
29
ge_proto_xxxx_PreRunAfterPrepare.txt
GE
Graph structure after all graph preparation processing (same as the previous graph).
Trustlisted
30
ge_proto_xxxx_OptimizeQuantGraph_FeGraphFusionAfter.txt
FE
Graph structure after quantization in the graph optimization phase.
-
31
ge_proto_xxxx_OptimizeOriginalGraph_FeGraphFusionAfter.txt
FE
Graph structure after graph fusion.
-
32
ge_proto_xxxx_OptimizeOriginalGraph_FeTopoSortingAfter.txt
FE
Graph structure after topological sorting following graph fusion, for detecting cycles.
-
33
ge_proto_xxxx_OptimizeOriginalGraph_DSAFeOpJudgeAfter.txt
FE
Graph after DataType and Format selection through dynamic shape analysis.
-
34
ge_proto_xxxx_HcclBeforeOptimizeOriginalGraph.txt
HCCL
Graph before HCCL-based original-graph optimization.
-
35
ge_proto_xxxx_HcclAfterOptimizeOriginalGraph.txt
HCCL
Graph after HCCL-based original-graph optimization.
-
36
ge_proto_xxxx_RunCustomPassAfterBuiltinFusionPass.txt
GE
Exit graph of the custom pass after built-in pass execution.
Trustlisted
37
ge_proto_xxxx_PreRunAfterOptimizeOriginalGraph.txt
GE
Graph structure after original-graph optimization via each operator information library (by calling the OptimizeOriginalGraph API).
Trustlisted
38
ge_proto_xxxx_PrepareAfterUpdateInputOutputByUserOptions.txt
GE
Graph structure after the graph inputs and outputs are processed based on user-specified options.
-
39
ge_proto_xxxx_PrepareAfterUpdateVariableFormats.txt
GE
Graph structure after variable Format processing.
-
40
ge_proto_xxxx_PreRunAfterPrepareRunningFormatRefiner.txt
GE
Same as the previous graph.
-
41
ge_proto_xxxx_BeforeOptimizeOriginalGraphJudgeInsert.txt
FE
Entry graph of the op_judge workflow.
-
42
ge_proto_xxxx_OptimizeOriginalGraph_FeOpDtypeJudgeAfter.txt
FE
Graph after precision mode selection.
-
43
ge_proto_xxxx_PreRunAfterRefineRunningPrecision.txt
GE
Graph after precision selection.
-
44
ge_proto_xxxx_AfterPrecisionRefine.txt
GE
Graph generated after conversion-operator fusion following precision selection.
-
45
ge_proto_xxxx_PreRunAfterAfterPrecisionRefine.txt
GE
Graph generated after conversion-operator fusion and automatic fusion following precision selection.
If automatic fusion is not enabled, this subgraph is identical to
ge_proto_xxxx_AfterPrecisionRefine.txt.
-
46
ge_proto_xxxx_OptimizeOriginalGraph_FeOpFormatJudgeAfter.txt
FE
Full op_judge graph after format selection.
-
47
ge_proto_xxxx_OptimizeOriginalGraph_FeDistHeavyFormatAfter.txt
FE
Graph structure after format propagation for heavy operators.
-
48
ge_proto_xxxx_OptimizeOriginalGraph_FeInsertTransNodeAfter.txt
FE
Graph structure after conversion operator insertion.
-
49
ge_proto_xxxx_PreRunAfterRefineRunningFormat.txt
GE
Graph structure after optimization via each operator information library (by calling the OptimizeOriginalGraphJudgeInsert API).
-
50
ge_proto_xxxx_PreRunAfterSubexpressionMigration.txt
GE
Graph structure after common subexpression extraction in dynamic-profile scenarios.
-
51
ge_proto_xxxx_before_SameTransdataBreadthFusionPass.txt
GE
Entry graph of SameTransdataBreadthFusionPass.
-
52
ge_proto_xxxx_after_SameTransdataBreadthFusionPass.txt
GE
Exit graph of SameTransdataBreadthFusionPass.
-
53
ge_proto_xxxx_OptimizeStage1_1.txt
GE
Graph structure after graph optimization stage 1_1.
-
54
ge_proto_xxxx_OptimizeStage1_2.txt
GE
Graph structure after graph optimization stage 1_2.
-
55
ge_proto_xxxx_PreRunAfterOptimize1.txt
GE
Graph structure after all graph optimization stage 1 processing.
-
56
ge_proto_xxxx_PreRunAfterOptimizeAfterStage1.txt
GE
Graph structure after optimization via each operator information library (by calling the OptimizeAfterStage1 API).
Trustlisted
57
ge_proto_xxxx_RunCustomPassAfterOriginGraphOptimize.txt
GE
Custom pass executed after original graph optimization.
Trustlisted
58
ge_proto_xxxx_PreRunAfterInferShape2.txt
GE
Graph structure after the second InferShape processing.
-
59
ge_proto_xxxx_BeforeStagePartition.txt
GE
Graph before stage partitioning.
-
60
ge_proto_xxxx_AfterStagePartition.txt
GE
Graph after stage partitioning.
-
61
ge_proto_xxxx_AfterEnginePlacer.txt
GE
Graph after engine selection.
-
62
ge_proto_xxxx_Before_DSP.txt
GE
Graph before static-dynamic partitioning.
-
63
ge_proto_xxxx_After_DSP.txt
GE
Graph after static-dynamic model partitioning.
-
64
ge_proto_xxxx_AfterDynamicShapePartition.txt
GE
Graph structure after dynamic-shape graph partitioning.
-
65
ge_proto_xxxx_MergedComputeGraphAfterCompositeEnginePartition.txt
GE
Structure of the merged graph after opposite subgraph partitioning and subgraph optimization.
-
66
ge_proto_xxxx_partition0_rank0_inputNodeGraph_AtomicEnginePartitioning.txt
GE
Structure of the input node subgraph after graph partitioning based on the atomic engine rules.
-
67
ge_proto_xxxx_partition0_rank1_new_sub_graph1_AtomicEnginePartitioning.txt
GE
Structure of subgraph 1 after graph partitioning based on the atomic engine rules.
-
68
ge_proto_xxxx_partition0_rank2_new_sub_graph110_AtomicEnginePartitioning.txt
GE
Structure of subgraph 110 after graph partitioning based on the atomic engine rules.
-
69
ge_proto_xxxx_OptimizeSubgraphPreProc.txt
GE
Exit graph after subgraph optimization preprocessing.
-
70
ge_proto_xxxx_DNN_VM_RTS_OptimizeSubGraphBefore.txt
RTS
-
-
71
ge_proto_xxxx_DNN_VM_RTS_OptimizeSubGraphAfter.txt
RTS
-
-
72
ge_proto_xxxx_AIcoreEngine_OptimizeSubGraphBefore.txt
FE
Entry graph before AI Core subgraph optimization.
-
73
ge_proto_xxxx_OptimizeSubGraphBefore.txt
GE
Subgraph structure before subgraph optimization. Each subgraph has a copy with the same name but different sequence numbers; the total count depends on the number of subgraphs.
-
74
ge_proto_xxxx_OptimizeSubGraphAfter.txt
GE
Subgraph structure after subgraph optimization. Each subgraph has a copy with the same name but different sequence numbers; the total count depends on the number of subgraphs.
-
75
ge_proto_xxxx_partition0_rank1_new_sub_graph1_lxfusion_input.txt
AOE(lxfusion)
SGAT input graph for ATC and AOE baseline scenarios.
-
76
ge_proto_xxxx_partition0_rank1_new_sub_graph1_after_rebuild.txt
AOE(lxfusion)
UB fusion graph within the AOE SGAT internal workflow.
-
77
ge_proto_xxxx_AIcoreEngine_OptimizeSubGraphAfter.txt
FE
Exit graph after AI Core subgraph optimization.
-
78
ge_proto_xxxx_OptimizeSubgraphPostProc.txt
GE
Exit graph after subgraph optimization postprocessing.
-
79
ge_proto_xxxx_mergedComputeGraph.txt
GE
Structure of the merged graph (same as the previous graph).
-
80
ge_proto_xxxx_MergedComputeGraphAfterAtomicEnginePartition.txt
GE
Structure of the merged graph after opposite atomic engine partitioning and subgraph optimization.
-
81
ge_proto_xxxx_PreRunAfterOptimizeSubgraph.txt
GE
Graph structure after subgraph optimization.
Trustlisted
82
ge_proto_xxxx_OptimizeWholeGraphaicpu_tf_optimizer.txt
GE
Graph information after original-graph optimization API calls by each engine (OptimizeWholeGraph followed by the engine name).
-
83
ge_proto_xxxx_OptimizeWholeGraphaicpu_ascend_optimizer.txt
GE
Graph information after original-graph optimization API calls by each engine (OptimizeWholeGraph followed by the engine name).
-
84
ge_proto_xxxx_OptimizeWholeGraphdvpp_graph_optimizer.txt
GE
Exit graph after DVPP optimization in full-graph optimization.
-
85
ge_proto_xxxx_OptimizeWholeGraphhccl_alltoallvc_fusion_optimizer.txt
HCCL
Graph after fusion optimization in the HCCL-based original-graph optimization phase.
-
86
ge_proto_xxxx_OptimizeWholeGraphAIcoreEngine.txt
GE
Graph information after original-graph optimization API calls by each engine (OptimizeWholeGraph followed by the engine name).
-
87
ge_proto_xxxx_OptimizeWholeGraphDSAEngine.txt
FE
Graph information after original-graph optimization API calls by each engine (OptimizeWholeGraph followed by the engine name).
-
88
ge_proto_xxxx_OptimizeWholeGraphhccl_graph_optimizer.txt
HCCL
Graph optimized in the HCCL-based original-graph optimization phase.
-
89
ge_proto_xxxx_OptimizeWholeGraphDNN_VM_RTS_GRAPH_OPTIMIZER_STORE.txt
GE
Graph information after original-graph optimization API calls by each engine (OptimizeWholeGraph followed by the engine name).
-
90
ge_proto_xxxx_OptimizeWholeGraphDNN_VM_RTS_FFTS_PLUS_GRAPH_OPTIMIZER_STORE.txt
RTS
Graph information after original-graph optimization API calls by each engine (OptimizeWholeGraph followed by the engine name).
-
91
ge_proto_xxxx_OptimizeWholeGraphDNN_VM_HOST_CPU_OPTIMIZER.txt
GE
Graph information after original-graph optimization API calls by each engine (OptimizeWholeGraph followed by the engine name).
-
92
ge_proto_xxxx_OptimizeWholeGraphge_local_graph_optimizer.txt
GE
Graph information after original-graph optimization API calls by each engine (OptimizeWholeGraph followed by the engine name).
-
93
ge_proto_xxxx_OptimizeWholeGraphffts_plus.txt
FE
Graph information after original-graph optimization API calls by each engine (OptimizeWholeGraph followed by the engine name).
-
94
ge_proto_xxxx_PreRunAfterOptimizeWholeGraph.txt
GE
Graph structure after optimization via each operator information library (by calling the OptimizeWholeGraph API).
-
95
ge_proto_xxxx_PreRunAfterOptimize2.txt
GE
Graph structure after all graph optimization stage 2 processing.
-
96
ge_proto_xxxx_PreRunAfterOptimizeGraphBeforeBuild.txt
GE
Entry graph for model build.
Trustlisted
97
ge_proto_xxxx_PreRunAfterOptimizeTensorMove.txt
GE
Graph after redundant TensorMove node optimization.
-
98
ge_proto_xxxx_BeforeHandleMemConflict.txt
GE
Graph before memory conflict resolution.
-
99
ge_proto_xxxx_AfterHandleMemConflict.txt
GE
Graph after memory conflict resolution.
-
100
ge_proto_xxxx_BeforeHandleMemoryLayoutConflict.txt
GE
Entry graph before memory layout conflict resolution.
-
101
ge_proto_xxxx_PreRunAfterMemConflictProc.txt
GE
Exit graph after memory read/write conflict resolution.
-
102
ge_proto_xxxx_partition0_rank0_inputNodeGraph_SecondPartitioning.txt
GE
Structure of the input node subgraph after second partitioning.
-
103
ge_proto_xxxx_partition0_rank1_new_sub_graph1_SecondPartitioning.txt
GE
Structure of subgraph 1 after second partitioning.
-
104
ge_proto_xxxx_partition0_rank2_new_sub_graph110_SecondPartitioning.txt
GE
Structure of subgraph 110 after second partitioning.
-
105
ge_proto_xxxx_BeforeUnfoldSubgraphs.txt
GE
Graph before dynamic-shape graph unfolding.
-
106
ge_proto_xxxx_AfterUnfoldSubgraphs.txt
GE
Graph after dynamic-shape graph unfolding.
-
107
ge_proto_xxxx_RunCustomPass_BeforeAssignLogicStream{pass_name}.txt
GE
Graph before user-defined stream allocation pass.
Trustlisted
108
ge_proto_xxxx_RunCustomPass_AfterAssignLogicStream{pass_name}.txt
GE
Graph after user-defined stream allocation pass.
Trustlisted
109
ge_proto_xxxx_AfterAssignResource.txt
GE
Graph after stream allocation, memory allocation, and GenTask.
-
110
ge_proto_xxxx_Build.txt
GE
Exit graph for model build.
Trustlisted
111
ge_proto_xxxx_PreRunAfterBuild.txt
GE
Same as the previous graph.
-
112
ge_proto_xxxx_BeforeAttrsCompress.txt
GE
Graph before offline model compression.
-
113
ge_proto_xxxx_AfterAttrsCompress.txt
GE
Graph after offline model compression.
-
114
ge_proto_xxxx_BeforeAttrsDecompress.txt
GE
Graph before offline model decompression.
-
115
ge_proto_xxxx_AfterAttrsDecompress.txt
GE
Graph after offline model decompression.
-
116
ge_proto_xxxx_ComputeGraphBeforeLowering.txt
GE
Computational graph before lowering.
Trustlisted
117
ge_proto_xxxx_Before_MultiStream_LoweringFirstLastEventSync.txt
GE
Execution graph before multi-stream event insertion.
-
118
ge_proto_xxxx_ExeGraphBeforeOptimize.txt
GE
Execution graph before graph optimization.
Trustlisted
119
ge_proto_xxxx_AfterZeroCopy.txt
GE
Execution graph after zero-copy optimization.
-
120
ge_proto_xxxx_AfterCEM.txt
GE
Execution graph after CEM optimization.
-
121
ge_proto_xxxx_AfterCopyFlowLaunch.txt
GE
Execution graph after inline copy optimization.
-
122
ge_proto_xxxx_TrustOutTensorAfter.txt
GE
Execution graph after TrustOutTensor optimization.
-
123
ge_proto_xxxx_AfterAicpuFuseHostInputs.txt
GE
Graph after AicpuFuseHostInputs optimization.
-
124
ge_proto_xxxx_ExecuteGraphAfterSplit.txt
GE
Final execution graph after dynamic-shape graph splitting.
Trustlisted
Applicable Models
All processors