Documentation

Graph Build Cache

Graph build cache supports disk persistence of graph build results. When an application runs again, it can directly load the build results cached on the disk. It also helps reduce the graph build duration.

Restrictions

  • The cache file is determined based on ge.graph_compiler_cache_dir and ge.graph_key in options. If the cache file does not exist, the cache is saved. If the cache file exists, the cache is directly loaded. This function takes effect only when both ge.graph_compiler_cache_dir and ge.graph_key are not empty.
  • The cache directory specified by ge.graph_compiler_cache_dir must exist. Otherwise, the build fails.
  • ge.graph_key must be unique. Otherwise, the cache saved in the same ge.graph_key is directly loaded.
  • In the multi-process scenario, if different graphs use the same graph key and cache dir, the graph cached later is not built. Instead, the graph cached earlier is loaded.
  • After a graph is changed, the original cache file is unavailable. You need to manually delete the cache file from the cache directory or modify ge.graph_key to rebuild and generate a cache file.
  • The cross-version cache cannot ensure compatibility. If the version is upgraded, clear the cache directory and recompile and generate the cache.

Procedure

1
2
3
4
5
6
std::map<ge::AscendString, ge::AscendString> session_options = {{"ge.graph_compiler_cache_dir", "./build_cache_dir"}};
std::shared_ptr<ge::Session> session = std::make_shared<ge::Session>(session_options);
const auto graph = CreateGraph();
std::map<ge::AscendString, ge::AscendString> graph_options = {{"ge.graph_key", "test_graph_001"}};
auto = session->AddGraph(0, graph, graph_options);
...

Cache File Generation Rules

The generated files include:

  • Model cache file
  • Index file, which helps users quickly find the corresponding cache file using graph_key. The following is an example of an index file:
     1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
    {
    "cache_file_list":[
        {
            "cache_file_name":"./cache_dir/graph_$key1_20230117202307.om",
            "graph_key":"graph_$key1",
            "var_desc_file_name":"./cache_dir/graph_$key1_20230117202307.rdcpkt"
        },
        {
            "cache_file_name":"./cache_dir/graph_$key1_20230117203007.om",
            "graph_key":"graph_$key1",
            "var_desc_file_name":"./cache_dir/graph_$key1_20230117203007.rdcpkt"
        }
    ]
}
  • Variable format file, which is generated only when variables exist in the graph. This parameter is used by the framework to match the model cache file. If the format of the variable in the graph corresponding to graph_key changes, the cache file cannot be directly restored. In this scenario, the building process is triggered again to generate a new cache file.
  • If the external weight function is enabled, that is, the ge.externalWeight parameter is set to 1 in options, the weight directory is generated in the path specified by the ge.graph_compiler_cache_dir parameter to store the weight information of the Const/Constant node on the original network.
File name generation rules:
  • The index file is named ge.graph_key.idx.
  • The model cache file is named ge.graph_keyTimestamp.om.
  • The variable file is named ge.graph_keyTimestamp.rdcpkt.