DataFlow Subgraph Compilation Cache

Function Description

After the DataFlow compilation cache function is enabled and when the subgraph of a single DataFlow node is modified, only the modified subgraph is incrementally compiled and other subgraphs use the cache, reducing the compilation time.

Constraints

This function supports FlowGraph but not AscendGraph.
Currently, models with resource operators are not supported.
Cache compatibility across versions is not guaranteed. After version upgrade, clear the cache directory and recompile to regenerate the cache.
After a DataFlow subgraph is changed, the original cache file is unavailable. You need to manually delete the cache file from the cache directory, and generate a cache file by recompilation.

How to Use

Enable the cache configuration.

A configuration example is provided as follows:

    # Perform initialization.
    SOC_VERSION = os.getenv('SOC_VERSION', 'AscendXXX')
    options = {
        "ge.exec.deviceId":"0",
        "ge.exec.logicalDeviceClusterDeployMode":"SINGLE",
        "ge.exec.logicalDeviceId":"[0:0]",
       "ge.graph_compiler_cache_dir":"./build_cache_dir"
    }
    df.init(options)

    flow_node0 = df.FlowNode(input_num=1, output_num=2)
    flow_node0_out = flow_node0(data0)

    # Construct a FlowGraph. Here focuses on the option usage. For details about how to construct a DataFlow graph, see other sections.
    options = {
        "ge.graph_key":"test_graph_00"
    }
    dag = df.FlowGraph([out for out in flow_node0_out], options)

**Table 1** Parameters
Parameter	Description
ge.graph_compiler_cache_dir	Disk cache directory for graph compilation. This parameter is used together with ge.graph_key. This function takes effect only when both ge.graph_compiler_cache_dir and ge.graph_key are not empty. The configured cache directory must exist. Otherwise, the compilation fails. After the graph is changed, the original cache files are unavailable. You need to manually delete the cache files (including the model cache file, index file, and variable format file) in the cache directory, and recompile the graph to generate the cache files.
ge.graph_key	Unique graph ID. It is recommended that the value contain a maximum of 128 characters, including only letters, digits (0–9), underscores (_), and hyphens (-). If the condition is not met, the system reports an error.

After compilation, the cache files are generated in the specified directory. For details, see Cache File Generation Rules.

If you need to recompile the graph after it changes, perform the following steps.
Graph changes include updates to the graph or config.json file corresponding to GraphProcessPoint, UDF implementation files, and deployment information.
Manually delete the cache files of the entire graph, including the model cache file, index file, and variable format file. For details, see Cache File Generation Rules.
Recompile the graph.
After compilation, the cache files are generated in the specified directory. For details, see Cache File Generation Rules.

Cache File Generation Rules

The generated files include:

Model cache file

Index file, which allows users to quickly find the corresponding cache file using graph_key. The following is an example of an index file:

{
    "cache_file_list":[
        {
            "cache_file_name":"./cache_dir/graph_$key1_20230117202307.om",
            "graph_key":"graph_$key1",
            "var_desc_file_name":"./cache_dir/graph_$key1_20230117202307.rdcpkt"
        },
        {
            "cache_file_name":"./cache_dir/graph_$key1_20230117203007.om",
            "graph_key":"graph_$key1",
            "var_desc_file_name":"./cache_dir/graph_$key1_20230117203007.rdcpkt"
        }
    ]
}

Variable format file, which is generated only when variables exist in the graph. This file is used by the framework to match the model cache file. If the format of the variable in the graph corresponding to graph_key changes, previously cached files cannot be directly restored and recompilation will be triggered to regenerate cache files in such cases.
weight directory, which is generated in the path specified by the ge.graph_compiler_cache_dir parameter to store the weight information of the Const/Constant node in the original network if the external weight function is enabled (that is, the ge.externalWeight parameter is set to 1 in options).
DataFlow subgraph cache folder, which is generated only in FlowGraph scenarios. Its name must contain the graph_key field. And the folder stores cache files generated during DataFlow subgraph compilation.

File name generation rules:

When the value of ge.graph_key contains only uppercase letters, lowercase letters, digits, underscores (_), and hyphens (-), and does not exceed 128 characters:
- The index file is named in the format of ge.graph_key.idx.
- The model cache file is named in the format of ge.graph_keyCurrent time.om.
- The variable format file is named in the format of ge.graph_keyCurrent time.rdcpkt.
If any of the preceding conditions is not met, the system will throw an error.

Parent topic: Special Topics