Dump Graph Details

Before model conversion, set the following environment variables to dump the model graphs generated during build. You can observe model changes by viewing the dump graphs.

export DUMP_GE_GRAPH=1      # Set the content level of the dump graph.
export DUMP_GRAPH_LEVEL=1   # Set the number of dump graphs.

The following files are generated in the current path where the atc command is executed. For details about the environment variables, see 2.

  • ge_onnx*.pbtxt: model description structure based on ONNX. You can open this file using visualizer software such as Netron.
  • ge_proto*.txt: text file stored in Protobuf format. You can convert it into a JSON file to facilitate fault locating. This file appears in pair with the ge_onnx*.pbtxt file, but has more attributes of the string type than the ge_onnx*.pbtxt file, making the ge_proto*.txt file more comprehensive. You can open either of them.

    Compared with the ge_onnx*.pbtxt file, the ge_proto*.txt file has a smaller size. Therefore, setting DUMP_GE_GRAPH to 2 or 3 has the same effect on the ge_proto*.txt file, that is, dumping information without data such as weights.

  • ge_readable*.txt: (optional) highly readable text file in Dynamo FX graph style. This type of file is generated only when the DUMP_GRAPH_FORMAT environment variable is set and the value contains readable.

Each of the preceding files corresponds to a step in the model build process, for example, the build begins with the ge_onnx_xxxx_PreRunBegin.pbtxt file and ends with the ge_onnx_xxxx_PreRunAfterBuild.pbtxt file. Each file contains all operators involved in the corresponding step. For details about the subgraphs at each dump phase, see Table 1. (The dump subgraphs may vary with models, but the main workflow remains consistent.)

Table 1 Description of dump graph details

No.

Subgraph

Component

Phase

Trustlisted or Not

1

ge_proto_xxxx_GraphPreRunBegin.txt

GE

Graph before build.

-

2

ge_proto_xxxx_AfterFlowGraphPartition.txt

GE

Graph after flow partitioning (flow partitioning is a partitioning mode used in DataFlow).

-

3

ge_proto_xxxx_AfterParallelPartitioner.txt

GE

Graph after pipeline parallel partitioning (pipeline here refers to PP in the backend inference scenario).

-

4

ge_proto_xxxx_PreRunBegin.txt

GE

Original graph structure.

Trustlisted

5

ge_proto_xxxx_RunCustomPassBeforeInfershape.txt

GE

Exit graph of the custom pass before InferShape.

Trustlisted

6

ge_proto_xxxx_PreRunAfterInitPreparation.txt

FE

Graph structure after all initialization processing in the graph preparation phase.

-

7

ge_proto_xxxx_PreRunAfterHandleSummaryOp.txt

GE

Graph structure after Summary node processing.

-

8

ge_proto_xxxx_PrepareAfterCheckAndUpdateInput.txt

GE

Graph structure after the graph input is checked and updated.

-

9

ge_proto_xxxx_PrepareAfterGraphEquivalentTransformation.txt

GE

Graph structure after equivalent replacement of For-loop graph structures with While-loop graph structures.

-

10

ge_proto_xxxx_PrepareAfterProcessOutput.txt

GE

Graph structure after graph data processing.

-

11

ge_proto_xxxx_PrepareAfterOptimizeAfterGraphNormalization.txt

GE

Exit graph after graph optimization following graph normalization.

-

12

ge_proto_xxxx_PrepareAfterInsertAipp.txt

GE

Graph structure after AIPP processing (when AIPP parameters are configured).

-

13

ge_proto_xxxx_PrepareAfterProcessAippNodesDataFormat.txt

GE

Exit graph after AIPP node format update.

-

14

ge_proto_xxxx_PreRunAfterNormalizeGraph.txt

GE

Exit graph after graph normalization.

Trustlisted

15

ge_proto_xxxx_PreRunAfterOptimizeGraphInit.txt

GE

Exit graph after graph optimization and initialization.

-

16

ge_proto_xxxx_OptimizeGraph_TagNoConstFoldingAfter.txt

FE

Used in quantization scenarios. The FE tags operators that should skip constant folding. During constant folding, the GE checks this tag and skips constant folding if the tag exists.

-

17

ge_proto_xxxx_HcclAfterOptimizeGraphPrepare.txt

HCCL

Graph after optimization in the HCCL graph preparation phase.

-

18

ge_proto_xxxx_PreRunAfterOptimizeGraphPrepare.txt

GE

Graph structure after original-graph preparation via each operator information library (by calling the OptimizeGraphPrepare API).

-

19

ge_proto_xxxx_PrepareAfterProcessBeforeInfershape.txt

GE

Graph structure after dead-edge elimination for conditional operators.

-

20

ge_proto_xxxx_AfterFirstInferformat.txt

GE

Graph structure after full-graph format inference.

-

21

ge_proto_xxxx_AfterInfershape.txt

GE

Graph structure after full-graph shape inference, accompanied by constant folding.

Trustlisted

22

ge_proto_xxxx_PrepareAfterInferFormatAndShape.txt

GE

Graph structure after all InferFormat and InferShape processing. A second full-graph InferFormat occurs between this and the previous graph.

-

23

ge_proto_xxxx_RunCustomPass_AfterInferShape.txt

GE

Exit graph of the custom pass after InferShape.

Trustlisted

24

ge_proto_xxxx_AfterSecondInferformat.txt

GE

Graph after the second format inference.

-

25

ge_proto_xxxx_PrepareAfterCtrlFlowPreProcess.txt

GE

Graph structure after conditional operator preprocessing.

-

26

ge_proto_xxxx_PrepareAfterGetDynamicOutputShape.txt

GE

Graph structure after graph output processing in dynamic-profile scenarios.

-

27

ge_proto_xxxx_PrepareAfterProcessAippStage2.txt

GE

Graph structure after graph input node processing in AIPP mode.

-

28

ge_proto_xxxx_PrepareAfterPrepareOptimize.txt

GE

Graph structure after optimization in the graph preparation phase.

-

29

ge_proto_xxxx_PreRunAfterPrepare.txt

GE

Graph structure after all graph preparation processing (same as the previous graph).

Trustlisted

30

ge_proto_xxxx_OptimizeQuantGraph_FeGraphFusionAfter.txt

FE

Graph structure after quantization in the graph optimization phase.

-

31

ge_proto_xxxx_OptimizeOriginalGraph_FeGraphFusionAfter.txt

FE

Graph structure after graph fusion.

-

32

ge_proto_xxxx_OptimizeOriginalGraph_FeTopoSortingAfter.txt

FE

Graph structure after topological sorting following graph fusion, for detecting cycles.

-

33

ge_proto_xxxx_OptimizeOriginalGraph_DSAFeOpJudgeAfter.txt

FE

Graph after DataType and Format selection through dynamic shape analysis.

-

34

ge_proto_xxxx_HcclBeforeOptimizeOriginalGraph.txt

HCCL

Graph before HCCL-based original-graph optimization.

-

35

ge_proto_xxxx_HcclAfterOptimizeOriginalGraph.txt

HCCL

Graph after HCCL-based original-graph optimization.

-

36

ge_proto_xxxx_RunCustomPassAfterBuiltinFusionPass.txt

GE

Exit graph of the custom pass after built-in pass execution.

Trustlisted

37

ge_proto_xxxx_PreRunAfterOptimizeOriginalGraph.txt

GE

Graph structure after original-graph optimization via each operator information library (by calling the OptimizeOriginalGraph API).

Trustlisted

38

ge_proto_xxxx_PrepareAfterUpdateInputOutputByUserOptions.txt

GE

Graph structure after the graph inputs and outputs are processed based on user-specified options.

-

39

ge_proto_xxxx_PrepareAfterUpdateVariableFormats.txt

GE

Graph structure after variable Format processing.

-

40

ge_proto_xxxx_PreRunAfterPrepareRunningFormatRefiner.txt

GE

Same as the previous graph.

-

41

ge_proto_xxxx_BeforeOptimizeOriginalGraphJudgeInsert.txt

FE

Entry graph of the op_judge workflow.

-

42

ge_proto_xxxx_OptimizeOriginalGraph_FeOpDtypeJudgeAfter.txt

FE

Graph after precision mode selection.

-

43

ge_proto_xxxx_PreRunAfterRefineRunningPrecision.txt

GE

Graph after precision selection.

-

44

ge_proto_xxxx_AfterPrecisionRefine.txt

GE

Graph generated after conversion-operator fusion following precision selection.

-

45

ge_proto_xxxx_PreRunAfterAfterPrecisionRefine.txt

GE

Graph generated after conversion-operator fusion and automatic fusion following precision selection.

If automatic fusion is not enabled, this subgraph is identical to

ge_proto_xxxx_AfterPrecisionRefine.txt.

-

46

ge_proto_xxxx_OptimizeOriginalGraph_FeOpFormatJudgeAfter.txt

FE

Full op_judge graph after format selection.

-

47

ge_proto_xxxx_OptimizeOriginalGraph_FeDistHeavyFormatAfter.txt

FE

Graph structure after format propagation for heavy operators.

-

48

ge_proto_xxxx_OptimizeOriginalGraph_FeInsertTransNodeAfter.txt

FE

Graph structure after conversion operator insertion.

-

49

ge_proto_xxxx_PreRunAfterRefineRunningFormat.txt

GE

Graph structure after optimization via each operator information library (by calling the OptimizeOriginalGraphJudgeInsert API).

-

50

ge_proto_xxxx_PreRunAfterSubexpressionMigration.txt

GE

Graph structure after common subexpression extraction in dynamic-profile scenarios.

-

51

ge_proto_xxxx_before_SameTransdataBreadthFusionPass.txt

GE

Entry graph of SameTransdataBreadthFusionPass.

-

52

ge_proto_xxxx_after_SameTransdataBreadthFusionPass.txt

GE

Exit graph of SameTransdataBreadthFusionPass.

-

53

ge_proto_xxxx_OptimizeStage1_1.txt

GE

Graph structure after graph optimization stage 1_1.

-

54

ge_proto_xxxx_OptimizeStage1_2.txt

GE

Graph structure after graph optimization stage 1_2.

-

55

ge_proto_xxxx_PreRunAfterOptimize1.txt

GE

Graph structure after all graph optimization stage 1 processing.

-

56

ge_proto_xxxx_PreRunAfterOptimizeAfterStage1.txt

GE

Graph structure after optimization via each operator information library (by calling the OptimizeAfterStage1 API).

Trustlisted

57

ge_proto_xxxx_RunCustomPassAfterOriginGraphOptimize.txt

GE

Custom pass executed after original graph optimization.

Trustlisted

58

ge_proto_xxxx_PreRunAfterInferShape2.txt

GE

Graph structure after the second InferShape processing.

-

59

ge_proto_xxxx_BeforeStagePartition.txt

GE

Graph before stage partitioning.

-

60

ge_proto_xxxx_AfterStagePartition.txt

GE

Graph after stage partitioning.

-

61

ge_proto_xxxx_AfterEnginePlacer.txt

GE

Graph after engine selection.

-

62

ge_proto_xxxx_Before_DSP.txt

GE

Graph before static-dynamic partitioning.

-

63

ge_proto_xxxx_After_DSP.txt

GE

Graph after static-dynamic model partitioning.

-

64

ge_proto_xxxx_AfterDynamicShapePartition.txt

GE

Graph structure after dynamic-shape graph partitioning.

-

65

ge_proto_xxxx_MergedComputeGraphAfterCompositeEnginePartition.txt

GE

Structure of the merged graph after opposite subgraph partitioning and subgraph optimization.

-

66

ge_proto_xxxx_partition0_rank0_inputNodeGraph_AtomicEnginePartitioning.txt

GE

Structure of the input node subgraph after graph partitioning based on the atomic engine rules.

-

67

ge_proto_xxxx_partition0_rank1_new_sub_graph1_AtomicEnginePartitioning.txt

GE

Structure of subgraph 1 after graph partitioning based on the atomic engine rules.

-

68

ge_proto_xxxx_partition0_rank2_new_sub_graph110_AtomicEnginePartitioning.txt

GE

Structure of subgraph 110 after graph partitioning based on the atomic engine rules.

-

69

ge_proto_xxxx_OptimizeSubgraphPreProc.txt

GE

Exit graph after subgraph optimization preprocessing.

-

70

ge_proto_xxxx_DNN_VM_RTS_OptimizeSubGraphBefore.txt

RTS

-

-

71

ge_proto_xxxx_DNN_VM_RTS_OptimizeSubGraphAfter.txt

RTS

-

-

72

ge_proto_xxxx_AIcoreEngine_OptimizeSubGraphBefore.txt

FE

Entry graph before AI Core subgraph optimization.

-

73

ge_proto_xxxx_OptimizeSubGraphBefore.txt

GE

Subgraph structure before subgraph optimization. Each subgraph has a copy with the same name but different sequence numbers; the total count depends on the number of subgraphs.

-

74

ge_proto_xxxx_OptimizeSubGraphAfter.txt

GE

Subgraph structure after subgraph optimization. Each subgraph has a copy with the same name but different sequence numbers; the total count depends on the number of subgraphs.

-

75

ge_proto_xxxx_partition0_rank1_new_sub_graph1_lxfusion_input.txt

AOE(lxfusion)

SGAT input graph for ATC and AOE baseline scenarios.

-

76

ge_proto_xxxx_partition0_rank1_new_sub_graph1_after_rebuild.txt

AOE(lxfusion)

UB fusion graph within the AOE SGAT internal workflow.

-

77

ge_proto_xxxx_AIcoreEngine_OptimizeSubGraphAfter.txt

FE

Exit graph after AI Core subgraph optimization.

-

78

ge_proto_xxxx_OptimizeSubgraphPostProc.txt

GE

Exit graph after subgraph optimization postprocessing.

-

79

ge_proto_xxxx_mergedComputeGraph.txt

GE

Structure of the merged graph (same as the previous graph).

-

80

ge_proto_xxxx_MergedComputeGraphAfterAtomicEnginePartition.txt

GE

Structure of the merged graph after opposite atomic engine partitioning and subgraph optimization.

-

81

ge_proto_xxxx_PreRunAfterOptimizeSubgraph.txt

GE

Graph structure after subgraph optimization.

Trustlisted

82

ge_proto_xxxx_OptimizeWholeGraphaicpu_tf_optimizer.txt

GE

Graph information after original-graph optimization API calls by each engine (OptimizeWholeGraph followed by the engine name).

-

83

ge_proto_xxxx_OptimizeWholeGraphaicpu_ascend_optimizer.txt

GE

Graph information after original-graph optimization API calls by each engine (OptimizeWholeGraph followed by the engine name).

-

84

ge_proto_xxxx_OptimizeWholeGraphdvpp_graph_optimizer.txt

GE

Exit graph after DVPP optimization in full-graph optimization.

-

85

ge_proto_xxxx_OptimizeWholeGraphhccl_alltoallvc_fusion_optimizer.txt

HCCL

Graph after fusion optimization in the HCCL-based original-graph optimization phase.

-

86

ge_proto_xxxx_OptimizeWholeGraphAIcoreEngine.txt

GE

Graph information after original-graph optimization API calls by each engine (OptimizeWholeGraph followed by the engine name).

-

87

ge_proto_xxxx_OptimizeWholeGraphDSAEngine.txt

FE

Graph information after original-graph optimization API calls by each engine (OptimizeWholeGraph followed by the engine name).

-

88

ge_proto_xxxx_OptimizeWholeGraphhccl_graph_optimizer.txt

HCCL

Graph optimized in the HCCL-based original-graph optimization phase.

-

89

ge_proto_xxxx_OptimizeWholeGraphDNN_VM_RTS_GRAPH_OPTIMIZER_STORE.txt

GE

Graph information after original-graph optimization API calls by each engine (OptimizeWholeGraph followed by the engine name).

-

90

ge_proto_xxxx_OptimizeWholeGraphDNN_VM_RTS_FFTS_PLUS_GRAPH_OPTIMIZER_STORE.txt

RTS

Graph information after original-graph optimization API calls by each engine (OptimizeWholeGraph followed by the engine name).

-

91

ge_proto_xxxx_OptimizeWholeGraphDNN_VM_HOST_CPU_OPTIMIZER.txt

GE

Graph information after original-graph optimization API calls by each engine (OptimizeWholeGraph followed by the engine name).

-

92

ge_proto_xxxx_OptimizeWholeGraphge_local_graph_optimizer.txt

GE

Graph information after original-graph optimization API calls by each engine (OptimizeWholeGraph followed by the engine name).

-

93

ge_proto_xxxx_OptimizeWholeGraphffts_plus.txt

FE

Graph information after original-graph optimization API calls by each engine (OptimizeWholeGraph followed by the engine name).

-

94

ge_proto_xxxx_PreRunAfterOptimizeWholeGraph.txt

GE

Graph structure after optimization via each operator information library (by calling the OptimizeWholeGraph API).

-

95

ge_proto_xxxx_PreRunAfterOptimize2.txt

GE

Graph structure after all graph optimization stage 2 processing.

-

96

ge_proto_xxxx_PreRunAfterOptimizeGraphBeforeBuild.txt

GE

Entry graph for model build.

Trustlisted

97

ge_proto_xxxx_PreRunAfterOptimizeTensorMove.txt

GE

Graph after redundant TensorMove node optimization.

-

98

ge_proto_xxxx_BeforeHandleMemConflict.txt

GE

Graph before memory conflict resolution.

-

99

ge_proto_xxxx_AfterHandleMemConflict.txt

GE

Graph after memory conflict resolution.

-

100

ge_proto_xxxx_BeforeHandleMemoryLayoutConflict.txt

GE

Entry graph before memory layout conflict resolution.

-

101

ge_proto_xxxx_PreRunAfterMemConflictProc.txt

GE

Exit graph after memory read/write conflict resolution.

-

102

ge_proto_xxxx_partition0_rank0_inputNodeGraph_SecondPartitioning.txt

GE

Structure of the input node subgraph after second partitioning.

-

103

ge_proto_xxxx_partition0_rank1_new_sub_graph1_SecondPartitioning.txt

GE

Structure of subgraph 1 after second partitioning.

-

104

ge_proto_xxxx_partition0_rank2_new_sub_graph110_SecondPartitioning.txt

GE

Structure of subgraph 110 after second partitioning.

-

105

ge_proto_xxxx_BeforeUnfoldSubgraphs.txt

GE

Graph before dynamic-shape graph unfolding.

-

106

ge_proto_xxxx_AfterUnfoldSubgraphs.txt

GE

Graph after dynamic-shape graph unfolding.

-

107

ge_proto_xxxx_RunCustomPass_BeforeAssignLogicStream{pass_name}.txt

GE

Graph before user-defined stream allocation pass.

Trustlisted

108

ge_proto_xxxx_RunCustomPass_AfterAssignLogicStream{pass_name}.txt

GE

Graph after user-defined stream allocation pass.

Trustlisted

109

ge_proto_xxxx_AfterAssignResource.txt

GE

Graph after stream allocation, memory allocation, and GenTask.

-

110

ge_proto_xxxx_Build.txt

GE

Exit graph for model build.

Trustlisted

111

ge_proto_xxxx_PreRunAfterBuild.txt

GE

Same as the previous graph.

-

112

ge_proto_xxxx_BeforeAttrsCompress.txt

GE

Graph before offline model compression.

-

113

ge_proto_xxxx_AfterAttrsCompress.txt

GE

Graph after offline model compression.

-

114

ge_proto_xxxx_BeforeAttrsDecompress.txt

GE

Graph before offline model decompression.

-

115

ge_proto_xxxx_AfterAttrsDecompress.txt

GE

Graph after offline model decompression.

-

116

ge_proto_xxxx_ComputeGraphBeforeLowering.txt

GE

Computational graph before lowering.

Trustlisted

117

ge_proto_xxxx_Before_MultiStream_LoweringFirstLastEventSync.txt

GE

Execution graph before multi-stream event insertion.

-

118

ge_proto_xxxx_ExeGraphBeforeOptimize.txt

GE

Execution graph before graph optimization.

Trustlisted

119

ge_proto_xxxx_AfterZeroCopy.txt

GE

Execution graph after zero-copy optimization.

-

120

ge_proto_xxxx_AfterCEM.txt

GE

Execution graph after CEM optimization.

-

121

ge_proto_xxxx_AfterCopyFlowLaunch.txt

GE

Execution graph after inline copy optimization.

-

122

ge_proto_xxxx_TrustOutTensorAfter.txt

GE

Execution graph after TrustOutTensor optimization.

-

123

ge_proto_xxxx_AfterAicpuFuseHostInputs.txt

GE

Graph after AicpuFuseHostInputs optimization.

-

124

ge_proto_xxxx_ExecuteGraphAfterSplit.txt

GE

Final execution graph after dynamic-shape graph splitting.

Trustlisted