DataFlow Subgraph Compilation Cache

Description

After the DataFlow compilation cache function is enabled and when the subgraph of a single DataFlow node is modified, only the modified subgraph is incrementally compiled, and other subgraphs use the cache, reducing the compilation time.

Constraints

This function supports FlowGraph but not AscendGraph.
Currently, models with resource operators are not supported.
Cache compatibility across versions is not guaranteed. After version upgrade, clear the cache directory and recompile to regenerate the cache.
After a DataFlow subgraph is changed, the original cache file is unavailable. You need to manually delete the cache file from the cache directory, and generate a cache file by recompilation.

Usage

Enable the cache configuration.

A configuration example is provided as follows:

std::map<ge::AscendString, ge::AscendString> session_options = {{"ge.graph_compiler_cache_dir", "./build_cache_dir"}};
std::shared_ptr<ge::Session> session = std::make_shared<ge::Session>(session_options);
const auto graph = CreateFlowGraph();
std::map<ge::AscendString, ge::AscendString> graph_options = {{"ge.graph_key", "test_graph_001"}};
auto = session->AddGraph(0, graph, graph_options);
...

**Table 1** Parameters
Parameter	Description
ge.graph_compiler_cache_dir	Disk cache directory for graph compilation. This parameter is used together with ge.graph_key. This function takes effect only when both ge.graph_compiler_cache_dir and ge.graph_key are not empty. The configured cache directory must exist. Otherwise, the compilation fails. After the graph is changed, the original cache files are unavailable. You need to manually delete the cache files (including the model cache file, index file, and variable format file) in the cache directory, and recompile the graph to generate the cache files. If you want to reduce the cache restoration time, you can add the cache.conf configuration file to the directory. The following is an example: { "cache_manual_check":true, "cache_debug_mode":true } cache_manual_check: indicates whether to enable the manual cache check mode. If it is set to true, you need to manually delete the corresponding subgraph cache files in the DataFlow subgraph cache folder when a graph is recompiled due to changes. cache_debug_mode: indicates whether to enable the cache debug mode. If it is set to true, the cache files of the entire graph, including the model cache file, index file, and variable format file, are not generated. For details, see Cache File Generation Rules.
ge.graph_key	Unique graph ID. It is recommended that the value contain a maximum of 128 characters, including only letters, digits (0–9), underscores (_), and hyphens (-). If the condition is not met, the system reports an error.

After the compilation is complete, the cache files are generated in the specified directory. For details, see Cache File Generation Rules.

If you need to recompile the graph after it changes, perform the following steps:
Graph changes include modifying the graph or config.json file corresponding to the graph ProcessPoint, modifying the UDF implementation file, and modifying the deployment information.

If cache.conf is configured for ge.graph_compiler_cache_dir and cache_debug_mode is set to true, you do not need to manually delete graph cache files.
1. (Optional) Manually delete the cache files of the entire graph, including the model cache file, index file, and variable format file. For details, see Cache File Generation Rules.
2. Recompile the graph.
  After the compilation is complete, the cache files are generated in the specified directory. For details, see Cache File Generation Rules.

Cache File Generation Rules

The generated files include:

Model cache file

Index file, which helps users quickly find the corresponding cache file using graph_key. The following is an example of an index file:

{
    "cache_file_list":[
        {
            "cache_file_name":"./cache_dir/graph_$key1_20230117202307.om",
            "graph_key":"graph_$key1",
            "var_desc_file_name":"./cache_dir/graph_$key1_20230117202307.rdcpkt"
        },
        {
            "cache_file_name":"./cache_dir/graph_$key1_20230117203007.om",
            "graph_key":"graph_$key1",
            "var_desc_file_name":"./cache_dir/graph_$key1_20230117203007.rdcpkt"
        }
    ]
}

Variable format file, which is generated only when variables exist in the graph. This file is used by the framework to match the model cache file. If the format of the variable in the graph corresponding to graph_key changes, the previous cache file cannot be directly restored. In this scenario, recompilation is triggered to generate new cache files.
If the external weight function is enabled, that is, the ge.externalWeight parameter is set to 1 in options, the weight directory is generated in the path specified by the ge.graph_compiler_cache_dir parameter to store the weight information of the Const/Constant node on the original network.
DataFlow subgraph cache folder, which is generated only in FlowGraph. Its name must contain the graph_key field. And the folder stores cache files generated during DataFlow subgraph compilation.

File name generation rules:

When the value of ge.graph_key contains a maximum of 128 characters, including only letters, digits (0–9), underscores (_), and hyphens (-):
- The index file is named in the format of ge.graph_key.idx.
- The model cache file is named in the format of ge.graph_keyCurrent time.om.
- The variable format file is named in the format of ge.graph_keyCurrent time.rdcpkt.
If any of the preceding conditions is not met, the system will throw an error.

Parent topic: Special Topics