ShardGraphs

Description

This API is not recommended. ShardGraphsToFile is recommended, which delivers better performance than ShardGraphs.

This API shards graphs in a session based on the AddGraph sequence. Sharded graphs are saved in the memory.

The sharding mode is specified by the ge.graphParallelOptionPath option. See Command-Line Options. If the parallel graph function is disabled, this API does not work.

Naming rules for sharded graphs:

In SPMD mode: The naming format is {Original graph name}_{ClusterId}_{ItemId}_{ChipId}_{VirtualStageId}_{Original GraphId}. The number of sharded graphs equals the number of original graphs multiplied by the number of shards.
In non-SPMD mode: The naming format is {Original graph name}_{Original GraphId}. The number of sharded graphs equals the number of original graphs.

After a graph is sharded, it does not exist in the session. For the ID of the newly generated graph after sharding, the original graph ID is replaced by another ID.

Differences between this API and ShardGraphsToFile:

ShardGraphs can search for the strategy, shard a graph, and output sharded graphs. Sharded graphs are saved in the memory and then flushed to disks using SaveGraphsToPb.
ShardGraphsToFile can search for the strategy, shard a graph, and output sharded graphs. Sharded graphs are also flushed to disks using this API (file_path must be a valid path).

Prototype

Status Session::ShardGraphs();

Restrictions

This API works only when the parallel graph function is enabled.

Parameters

N/A

Returns

Parameter	Type	Description
-	Status	SUCCESS: success. FAILED: failure.

Parameter

Type

Description

Status

SUCCESS: success.

FAILED: failure.

Example

Session session(options); // The parallel graph function is enabled through options.
Graph init_graph("init_graph");
Graph first_graph("first_graph");
Graph second_graph("second_graph");
session.ShardGraphs();

Parent topic: Class Session