DataFlow Offline Compilation

Description

DataFlow offline compilation refers to compilation in the development environment and loading and deployment in the operating environment. In this way, the compilation and link setup between the host and device are decoupled.

How to Use

Enable graph compilation caching. For example:

std::map<ge::AscendString, ge::AscendString> session_options = {{"ge.graph_compiler_cache_dir", "./build_cache_dir"}};
std::shared_ptr<ge::Session> session = std::make_shared<ge::Session>(session_options);
const auto graph = CreateFlowGraph();
std::map<ge::AscendString, ge::AscendString> graph_options = {{"ge.graph_key", "test_graph_001"}};
auto = session->AddGraph(0, graph, graph_options);
...

Configure the development environment.

Use the environment variable RESOURCE_CONFIG_PATH to configure the numa_config.json file in the target execution environment. The following is an example.

 export RESOURCE_CONFIG_PATH=numa_config.json// Specify the path for storing the heterogeneous resource description file.

For details about numa_config.json, see numa_config.json Configuration. The ipaddr field can be set to any IP address.

{
                    "host":{
                         "resourceType": "X86",
                    .........
                    "devList":[
                      {
                      "ipaddr":"XX.XX.XX.XX",     
                      "port":2509,
                      "deviceIdList":[0,1],
                      "resourceType":"Ascend",
                      "token":"OKIJBNHYGFVT7RGH",
                      "chip_count":2,
                      }
                   ],
}

**Table 1** Parameters
Parameter	Description
resourceType	Optional. Resource type. The value can be Ascend. It must be the same as the value of resourceType in the actual operating environment.
chip_count	Required. Count of accelerator cards (chips). It must be the same as the number of the deployed operating environments. If they are not the same, the system performs the following operations: An error is reported in the deployment phase, when the value of chip_count is greater than the number of deployed operating environments. The model is deployed based on load balancing in the compilation phase, when the value of chip_count is less than the number of deployed operating environments and the deployment location is not specified for load balancing. The extra devices in the operating environment will not be deployed. If this parameter is not configured, the offline function does not take effect.

Generate a graph.

Set the option ge.runFlag to 0 when calling the AddGraph API.

Call the BuildGraph API to compile the graph and generate a cache file.

For example:

map<AscendString, AscendString> options = {{"ge.graph_compiler_cache_dir", "./build_cache_dir"}, {"ge.runFlag", "0"}};
Session session(options);
map<AscendString, AscendString> graph_options = {{"ge.graph_key", "graph_key1"}}
session.AddGraph(1, g1, graph_options);
std::vector<InputTensorInfo> inputs;
auto ret = session.BuildGraph(1, inputs);
//The BuildGraph API can be replaced with the FeedDataFlowGraph API.
auto ret = session.FeedDataFlowGraph(1, inputs, dataInfo);

Offline deployment
Copy the model cache file, index file, and variable format file from graph_compiler_cache_dir in the development environment to graph_compiler_cache_dir in the operating environment. For details about the files, see Cache File Generation Rules.

The numa_config file in the execution environment must be the same as that in the compilation environment, except for the ipaddr field.

Parent topic: Special Topics