Documentation

DUMP_GRAPH_LEVEL

Description

DUMP_GRAPH_LEVEL sets the graph to dump.

This environment variable can be configured in either of the following ways. Both of them are used to control the number of flushed graphs. You can use them as required. Note that the two configuration methods cannot be used together.

  • Set it to a value. Possible values are as follows:
    • 1: dumps graphs in all phases.
    • 2: dumps graphs in the trustlist phase. For details about the trustlisted graphs, see the Trustlisted or Not column in Table 1.
    • 3: dumps the last generated graph, that is, the graph optimized and compiled by the GE.
    • 4: dumps the earliest generated graph, that is, the build entry graph provided to the software stack after the GE parses and maps operators. This graph structure has not been compiled or optimized by the GE.
  • Set it to a character string separated by vertical bars (|):

    For example, if this environment variable is set to aa|bb, graphs whose names contain aa and bb are dumped. aa and bb must be set to valid character strings during graph build. Valid strings can be obtained from a full dump of the graphs.

DUMP_GRAPH_LEVEL takes effect only when DUMP_GE_GRAPH is enabled. The default value is 2.

Example

  • Set this environment variable to a value: 
    export DUMP_GRAPH_LEVEL=1
  • Set this environment variable to a character string separated by vertical bars (|): 
    export DUMP_GRAPH_LEVEL="PreRunBegin|AfterInfershape"

Restrictions

  • If this environment variable is set to an invalid value, undefined behavior may occur.
  • If the operator dump data collection function is enabled, see "ge.exec.enableDump" in the Graph Mode Development Guide. Even if the DUMP_GRAPH_LEVEL environment variable is not configured or export DUMP_GRAPH_LEVEL="PreRunBegin|AfterInfershape" is configured but does not contain the Build character string, the subgraph ge_proto_xxxx_Build.txt is dumped.
  • This environment variable must be used together with DUMP_GE_GRAPH. That is, when DUMP_GE_GRAPH is enabled, DUMP_GRAPH_LEVEL can be used to control the generated dump graphs. For details about the full dump graph information, see Table 1.
    Table 1 Description of dump graph details

    Subgraph

    Phase

    Trustlisted or Not

    ge_proto_xxxx_GraphPreRunBegin.txt

    Graph before build.

    -

    ge_proto_xxxx_AfterFlowGraphPartition.txt

    Graph after flow partitioning (flow partitioning is a partitioning mode used in the data flow).

    -

    ge_proto_xxxx_AfterParallelPartitioner.txt

    Graph after pipeline parallel partitioning (pipeline here refers to PP in the backend inference scenario).

    -

    ge_proto_xxxx_PreRunBegin.txt

    Original graph structure.

    Trustlisted

    ge_proto_xxxx_RunCustomPassBeforeInfershape.txt

    Custom pass exit graph before InferShape.

    Trustlisted

    ge_proto_xxxx_PreRunAfterInitPreparation.txt

    Graph structure obtained after all initialization in the graph preparation phase.

    -

    ge_proto_xxxx_PreRunAfterHandleSummaryOp.txt

    Graph structure obtained after Summary node processing.

    -

    ge_proto_xxxx_PrepareAfterCheckAndUpdateInput.txt

    Graph structure obtained after the graph input is checked and updated.

    -

    ge_proto_xxxx_PrepareAfterGraphEquivalentTransformation.txt

    Graph structure obtained after the for loop is equivalently transformed into while loop.

    -

    ge_proto_xxxx_PrepareAfterProcessOutput.txt

    Graph structure obtained after graph data processing.

    -

    ge_proto_xxxx_PrepareAfterOptimizeAfterGraphNormalization.txt

    Optimization exit graph after graph normalization.

    -

    ge_proto_xxxx_PrepareAfterInsertAipp.txt

    Graph structure obtained after AIPP processing.

    -

    ge_proto_xxxx_PrepareAfterProcessAippNodesDataFormat.txt

    Exit graph after AIPP node format update.

    -

    ge_proto_xxxx_PreRunAfterNormalizeGraph.txt

    Exit graph after graph normalization.

    Trustlisted

    ge_proto_xxxx_PreRunAfterOptimizeGraphInit.txt

    Exit graph after graph optimization and initialization.

    -

    ge_proto_xxxx_OptimizeGraph_TagNoConstFoldingAfter.txt

    Used for quantization scenarios. The FE adds a tag for an operator to indicate no constant folding. When the GE checks that the tag exists, it does not perform constant folding.

    -

    ge_proto_xxxx_HcclAfterOptimizeGraphPrepare.txt

    Graph obtained after optimization in the HCCL graph preparation phase.

    -

    ge_proto_xxxx_PreRunAfterOptimizeGraphPrepare.txt

    Graph structure obtained after the original graphs in the operator information library are prepared (by calling the OptimizeGraphPrepare API).

    -

    ge_proto_xxxx_PrepareAfterProcessBeforeInfershape.txt

    Graph structure obtained after "dead edges"of the conditional operator are eliminated.

    -

    ge_proto_xxxx_AfterFirstInferformat.txt

    Graph structure obtained after format inference on the entire graph.

    -

    ge_proto_xxxx_AfterInfershape.txt

    Graph structure obtained after shape inference on the entire graph, with constant folding.

    Trustlisted

    ge_proto_xxxx_PrepareAfterInferFormatAndShape.txt

    Graph structure obtained after InferFormat and InferShape processing. This graph has undergone the second InferFormat processing, compared with the preceding graph.

    -

    ge_proto_xxxx_RunCustomPass_AfterInferShape.txt

    Custom pass exit graph after InferShape.

    Trustlisted

    ge_proto_xxxx_AfterSecondInferformat.txt

    Graph obtained after the second format inference.

    -

    ge_proto_xxxx_PrepareAfterCtrlFlowPreProcess.txt

    Graph structure obtained after the conditional operator is preprocessed.

    -

    ge_proto_xxxx_PrepareAfterGetDynamicOutputShape.txt

    Graph structure obtained after graph output processing in the presence of dynamic batch size profiles.

    -

    ge_proto_xxxx_PrepareAfterProcessAippStage2.txt

    Graph structure obtained after graph input processing in AIPP mode.

    -

    ge_proto_xxxx_PrepareAfterPrepareOptimize.txt

    Graph structure obtained after optimization in the graph preparation phase.

    -

    ge_proto_xxxx_PreRunAfterPrepare.txt

    Graph structure obtained after graph preparation, the same as the preceding graph.

    Trustlisted

    ge_proto_xxxx_OptimizeQuantGraph_FeGraphFusionAfter.txt

    Graph structure obtained after quantization in the graph optimization phase ends.

    -

    ge_proto_xxxx_OptimizeOriginalGraph_FeGraphFusionAfter.txt

    Graph structure obtained after graph fusion ends.

    -

    ge_proto_xxxx_OptimizeOriginalGraph_FeTopoSortingAfter.txt

    Graph structure obtained after graph fusion and topology sorting, for checking whether a ring is formed.

    -

    ge_proto_xxxx_OptimizeOriginalGraph_DSAFeOpJudgeAfter.txt

    Graph with DataType and Format selected after dynamic shape analysis.

    -

    ge_proto_xxxx_HcclBeforeOptimizeOriginalGraph.txt

    Graph before optimization of the original HCCL graph.

    -

    ge_proto_xxxx_HcclAfaterOptimizeOriginalGraph.txt

    Graph after optimization of the original HCCL graph.

    -

    ge_proto_xxxx_RunCustomPassAfterBuiltinFusionPass.txt

    Custom pass exit graph after internal pass.

    Trustlisted

    ge_proto_xxxx_PreRunAfterOptimizeOriginalGraph.txt

    Graph structure obtained after the original graphs in the operator information library are optimized (by calling the OptimizeOriginalGraph API).

    Trustlisted

    ge_proto_xxxx_PrepareAfterUpdateInputOutputByUserOptions.txt

    Graph structure obtained after the graph input and output are processed based on the user's command-line options.

    -

    ge_proto_xxxx_PrepareAfterUpdateVariableFormats.txt

    Graph structure obtained after the variable formats are processed.

    -

    ge_proto_xxxx_PreRunAfterPrepareRunningFormatRefiner.txt

    Same as the preceding graph.

    -

    ge_proto_xxxx_BeforeOptimizeOriginalGraphJudgeInsert.txt

    Entry graph of the operator judging process.

    -

    ge_proto_xxxx_OptimizeOriginalGraph_FeOpDtypeJudgeAfter.txt

    Graph after precision mode selection.

    -

    ge_proto_xxxx_PreRunAfterRefineRunningPrecision.txt

    Graph after precision selection.

    -

    ge_proto_xxxx_OptimizeOriginalGraph_FeOpFormatJudgeAfter.txt

    Graph with complete op_judge after format selection.

    -

    ge_proto_xxxx_OptimizeOriginalGraph_FeDistHeavyFormatAfter.txt

    Graph structure obtained after diffusion of heavy operators.

    -

    ge_proto_xxxx_OptimizeOriginalGraph_FeInsertTransNodeAfter.txt

    Graph structure obtained after the transform operator is inserted.

    -

    ge_proto_xxxx_PreRunAfterRefineRunningFormat.txt

    Graph structure obtained after each operator information library is optimized (by calling the OptimizeOriginalGraphJudgeInsert API).

    -

    ge_proto_xxxx_PreRunAfterSubexpressionMigration.txt

    Graph structure after the common subexpression is extracted in the scenario of dynamic dimension size profiles.

    -

    ge_proto_xxxx_before_SameTransdataBreadthFusionPass.txt

    SameTransdataBreadthFusionPass entry graph.

    -

    ge_proto_xxxx_after_SameTransdataBreadthFusionPass.txt

    SameTransdataBreadthFusionPass exit graph.

    -

    ge_proto_xxxx_OptimizeStage1_1.txt

    Graph structure obtained after graph optimization stage 1_1.

    -

    ge_proto_xxxx_OptimizeStage1_2.txt

    Graph structure obtained after graph optimization stage 1_2.

    -

    ge_proto_xxxx_PreRunAfterOptimize1.txt

    Graph structure obtained after optimization stage 1 for all graphs.

    -

    ge_proto_xxxx_PreRunAfterOptimizeAfterStage1.txt

    Graph structure obtained after each operator information library is optimized (by calling the OptimizeAfterStage1 API).

    Trustlisted

    ge_proto_xxxx_PreRunAfterInferShape2.txt

    Graph structure obtained after the second InferShape processing.

    -

    ge_proto_xxxx_BeforeStagePartition.txt

    Graph before stage partitioning.

    -

    ge_proto_xxxx_AfterStagePartition.txt

    Graph after stage partitioning.

    -

    ge_proto_xxxx_AfterEnginePlacer.txt

    Graph after engine selection.

    -

    ge_proto_xxxx_Before_DSP.txt

    Graph before partitioning static and dynamic models.

    -

    ge_proto_xxxx_After_DSP.txt

    Graph after partitioning static and dynamic models.

    -

    ge_proto_xxxx_AfterDynamicShapePartition.txt

    Graph structure obtained after graph partitioning with a dynamic shape.

    -

    ge_proto_xxxx_MergedComputeGraphAfterCompositeEnginePartition.txt

    Structure of the merged graph obtained after opposite subgraph partitioning and subgraph optimization.

    -

    ge_proto_xxxx_partition0_rank0_inputNodeGraph_AtomicEnginePartitioning.txt

    Structure of the input node subgraph obtained after graph partitioning based on the atomic engine rules.

    -

    ge_proto_xxxx_partition0_rank1_new_sub_graph1_AtomicEnginePartitioning.txt

    Structure of subgraph 1 obtained after graph partitioning based on the atomic engine rules.

    -

    ge_proto_xxxx_partition0_rank2_new_sub_graph110_AtomicEnginePartitioning.txt

    Structure of subgraph 110 obtained after graph partitioning based on the atomic engine rules.

    -

    ge_proto_xxxx_OptimizeSubgraphPreProc.txt

    Exit graph after subgraph optimization preprocessing.

    -

    ge_proto_xxxx_DNN_VM_RTS_OptimizeSubGraphBefore.txt

    -

    -

    ge_proto_xxxx_DNN_VM_RTS_OptimizeSubGraphAfter.txt

    -

    -

    ge_proto_xxxx_AIcoreEngine_OptimizeSubGraphBefore.txt

    Entry graph before AI Core subgraph optimization.

    -

    ge_proto_xxxx_OptimizeSubGraphBefore.txt

    Subgraph structure obtained before optimization. Subgraphs have the same name but different sequence numbers, depending on the number of subgraphs.

    -

    ge_proto_xxxx_OptimizeSubGraphAfter.txt

    Subgraph structure obtained after optimization. Subgraphs have the same name but different sequence numbers, depending on the number of subgraphs.

    -

    ge_proto_xxxx_partition0_rank1_new_sub_graph1_lxfusion_input.txt

    SGAT input graph in the ATC and AOE baseline scenarios.

    -

    ge_proto_xxxx_partition0_rank1_new_sub_graph1_after_rebuild.txt

    UB fusion graph of the AOE SGAT internal process.

    -

    ge_proto_xxxx_AIcoreEngine_OptimizeSubGraphAfter.txt

    Exit graph after AI Core subgraph optimization.

    -

    ge_proto_xxxx_OptimizeSubgraphPostProc.txt

    Exit graph after subgraph optimization postprocessing.

    -

    ge_proto_xxxx_mergedComputeGraph.txt

    Structure of the merged graph, the same as the preceding graph.

    -

    ge_proto_xxxx_MergedComputeGraphAfterAtomicEnginePartition.txt

    Structure of the merged graph obtained after opposite subgraph partitioning and subgraph optimization.

    -

    ge_proto_xxxx_PreRunAfterOptimizeSubgraph.txt

    Subgraph structure obtained after optimization.

    Trustlisted

    ge_proto_xxxx_OptimizeWholeGraphaicpu_tf_optimizer.txt

    Graph information obtained after the original graph optimization API of each engine is called. OptimizeWholeGraph is followed by the engine name.

    -

    ge_proto_xxxx_OptimizeWholeGraphaicpu_ascend_optimizer.txt

    Graph information obtained after the original graph optimization API of each engine is called. OptimizeWholeGraph is followed by the engine name.

    -

    ge_proto_xxxx_OptimizeWholeGraphdvpp_graph_optimizer.txt

    Exit graph after entire graph optimization and DVPP optimization.

    -

    ge_proto_xxxx_OptimizeWholeGraphhccl_alltoallvc_fusion_optimizer.txt

    Graph after fusion optimization in the original HCCL graph optimization phase.

    -

    ge_proto_xxxx_OptimizeWholeGraphAIcoreEngine.txt

    Graph information obtained after the original graph optimization API of each engine is called. OptimizeWholeGraph is followed by the engine name.

    -

    ge_proto_xxxx_OptimizeWholeGraphDSAEngine.txt

    Graph information obtained after the original graph optimization API of each engine is called. OptimizeWholeGraph is followed by the engine name.

    -

    ge_proto_xxxx_OptimizeWholeGraphhccl_graph_optimizer.txt

    Graph optimized in the original HCCL graph optimization phase.

    -

    ge_proto_xxxx_OptimizeWholeGraphDNN_VM_RTS_GRAPH_OPTIMIZER_STORE.txt

    Graph information obtained after the original graph optimization API of each engine is called. OptimizeWholeGraph is followed by the engine name.

    -

    ge_proto_xxxx_OptimizeWholeGraphDNN_VM_RTS_FFTS_PLUS_GRAPH_OPTIMIZER_STORE.txt

    Graph information obtained after the original graph optimization API of each engine is called. OptimizeWholeGraph is followed by the engine name.

    -

    ge_proto_xxxx_OptimizeWholeGraphDNN_VM_HOST_CPU_OPTIMIZER.txt

    Graph information obtained after the original graph optimization API of each engine is called. OptimizeWholeGraph is followed by the engine name.

    -

    ge_proto_xxxx_OptimizeWholeGraphge_local_graph_optimizer.txt

    Graph information obtained after the original graph optimization API of each engine is called. OptimizeWholeGraph is followed by the engine name.

    -

    ge_proto_xxxx_OptimizeWholeGraphffts_plus.txt

    Graph information obtained after the original graph optimization API of each engine is called. OptimizeWholeGraph is followed by the engine name.

    -

    ge_proto_xxxx_PreRunAfterOptimizeWholeGraph.txt

    Graph structure obtained after each operator information library is optimized (by calling the OptimizeWholeGraph API).

    -

    ge_proto_xxxx_PreRunAfterOptimize2.txt

    Graph structure obtained after optimization stage 2 for all graphs.

    -

    ge_proto_xxxx_PreRunAfterOptimizeGraphBeforeBuild.txt

    Entry graph for model build.

    Trustlisted

    ge_proto_xxxx_BeforeHandleMemConflict.txt

    Graph before the memory conflict is handled.

    -

    ge_proto_xxxx_AfterHandleMemConflict.txt

    Graph after the memory conflict is handled.

    -

    ge_proto_xxxx_BeforeHandleMemoryLayoutConflict.txt

    Entry graph before the memory layout conflict is handled.

    -

    ge_proto_xxxx_PreRunAfterMemConflictProc.txt

    Exit graph after the memory read/write conflict is handled.

    -

    ge_proto_xxxx_partition0_rank0_inputNodeGraph_SecondPartitioning.txt

    Structure of the input node subgraph obtained after second partitioning.

    -

    ge_proto_xxxx_partition0_rank1_new_sub_graph1_SecondPartitioning.txt

    Structure of subgraph 1 obtained after second partitioning.

    -

    ge_proto_xxxx_partition0_rank2_new_sub_graph110_SecondPartitioning.txt

    Structure of subgraph 110 obtained after second partitioning.

    -

    ge_proto_xxxx_BeforeUnfoldSubgraphs.txt

    Graph before dynamic-shape graph expansion.

    -

    ge_proto_xxxx_AfterUnfoldSubgraphs.txt

    Graph after dynamic-shape graph expansion.

    -

    ge_proto_xxxx_RunCustomPass_BeforeAssignLogicStream*.txt

    Graph before custom stream allocation pass processing. * indicates the specific pass name.

    Trustlisted

    ge_proto_xxxx_RunCustomPass_AfterAssignLogicStream*.txt

    Graph after custom stream allocation pass processing. * indicates the specific pass name.

    Trustlisted

    ge_proto_xxxx_AfterAssignResource.txt

    Graph after stream allocation, memory allocation, and GenTask.

    -

    ge_proto_xxxx_Build.txt

    Exit graph for model build.

    Trustlisted

    ge_proto_xxxx_PreRunAfterBuild.txt

    Same as the preceding graph.

    -

    ge_proto_xxxx_BeforeAttrsCompress.txt

    Graph before offline model compression.

    -

    ge_proto_xxxx_AfterAttrsCompress.txt

    Graph after offline model compression.

    -

    ge_proto_xxxx_BeforeAttrsDecompress.txt

    Graph before offline model decompression.

    -

    ge_proto_xxxx_AfterAttrsDecompress.txt

    Graph after offline model decompression.

    -

    ge_proto_xxxx_ComputeGraphBeforeLowering.txt

    Computational graph before lowering.

    Trustlisted

    ge_proto_xxxx_Before_MultiStream_LoweringFirstLastEventSync.txt

    Execution graph before multi-stream event insertion.

    -

    ge_proto_xxxx_ExeGraphBeforeOptimize.txt

    Execution graph before graph optimization.

    Trustlisted

    ge_proto_xxxx_AfterZeroCopy.txt

    Execution graph after zero-copy optimization.

    -

    ge_proto_xxxx_AfterCEM.txt

    Execution graph after CEM optimization.

    -

    ge_proto_xxxx_AfterCopyFlowLaunch.txt

    Execution graph after in-line copy optimization.

    -

    ge_proto_xxxx_TrustOutTensorAfter.txt

    Execution graph after TrustOutTensor optimization.

    -

    ge_proto_xxxx_ AicpuFuseHostInputsAfter.txt

    Graph after AicpuFuseHostInputs optimization.

    -

    ge_proto_xxxx_ExecuteGraphAfterSplit.txt

    Final execution graph of the dynamic shape.

    Trustlisted

Applicability

Atlas training products

Atlas inference products

Atlas A2 training products / Atlas A2 inference products

Atlas A3 training products / Atlas A3 inference products

Atlas 200I/500 A2 inference products

Parent topic: Graph Building
This site uses cookies to improve your browsing experience. By continuing to browse this site, you agree to our use of cookies.View Details