ENABLE_DYNAMIC_SHAPE_MULTI_STREAM
Description
During graph execution, enabling the multi-stream concurrency function can improve network performance in certain scenarios. Currently, the multi-stream concurrency function is disabled by default. If you want to enable this function in dynamic shape scenario, you can use this environment variable to enable it.
- 0: (default) disables the function.
- 1: enables the function.
Example
The multi-stream concurrency function is disabled by default. To enable this function, set this environment variable to 1.
export ENABLE_DYNAMIC_SHAPE_MULTI_STREAM=1
Restrictions
If a static subgraph exists in a dynamic-shape graph, the single stream configuration in the graph build config takes precedence over the environment variable ENABLE_DYNAMIC_SHAPE_MULTI_STREAM.
In other words, if single-stream execution is enabled for config in the PyTorch graph mode (TorchAir) and the environment variable for multi-stream execution is enabled, the former prevails. The following is a configuration example of the graph mode:
config.ge_config.enable_single_stream = True