文档

MemoryConfig Constructor

Description

Constructs an object of class MemoryConfig for configuring system memory handling.

Prototype

def __init__(self,
atomic_clean_policy=0,
static_memory_policy=0
)

Options

Option

Input/Output

Description

atomic_clean_policy

Input
Whether to clean up the memory occupied by all operators with the memset attribute (memset operators) on the network. The options are as follows:
  • 0 (default): enabled.
  • 1: disabled. Memory used by each memset operator is cleaned up separately. When the memset operators on the network occupy too much memory, you can try this method. However, this method may cause performance loss.

static_memory_policy

Input
Memory allocation mode used during network running.
  • 0: dynamic memory allocation. Memory is dynamically allocated based on the actual size.
  • 2: dynamic memory expansion supported by only static shape. In training and online inference scenarios, this environment variable can be used to implement memory overcommitment between multiple graphs in the same session. That is, the memory required by the maximum graph is allocated. For example, if the memory required by the current graph exceeds the memory of the previous graph, the memory of the previous graph is directly released. The memory is reallocated based on the memory required by the current graph.
  • 3: dynamic memory expansion supported by only dynamic shape, which solves the fragment problem during dynamic memory allocation and reduces the memory usage of the dynamic-shape network.
  • 4: dynamic memory expansion supported by both static and dynamic shapes.

The default value is 0.

NOTE:
  • This option cannot be set to 2 or 4 when multiple graphs are executed concurrently.
  • To be compatible with earlier versions, the system adopts the method of mode 2 even if this option is set to 1.
  • If this parameter is set to 3 or 4, memory gains are generated, but performance loss may occur.

Returns

An object of the MemoryConfig class, as an argument passed to the NPURunConfig call.

Restrictions

None

Examples

1
2
3
4
5
from npu_bridge.npu_init import *
...
mem_config = MemoryConfig(atomic_clean_policy=0, static_memory_policy=0)
session_config=tf.ConfigProto(allow_soft_placement=True)
config = NPURunConfig(memory_config=mem_config, session_config=session_config)