Mbuf Multi-Memory

Function Description

Mbuf is a shared memory that contains MbufHead and MbufData, as described in Table 1.

**Table 1** Introduction to Mbuf
Data type	Description
MbufHead	Data header, which contains the following types of data: Basic data content, such as DataType, DataSize, and Shape. Key information involved in data transfer: framework-related information such as retCode and routeLabel. Other configurable fields, such as starttime, endtime, and userdata.
MbufData	A memory segment inside the Mbuf that holds the actual valid data.

In DataFlow memory allocation scenarios, Mbuf serves as the unified memory management mechanism provided by the driver. All FlowMsg instances are created and recycled in the Mbuf pool. Each FlowMsg occupies one Mbuf memory segment, whose size is calculated as follows:

sizeMbuf  = 
       sizeof(MbufData) // Memory size of the actual valid data contained in FlowMsg
     + sizeof(MbufHead) // Size of the MbufHead. The actual size is 1,024 bytes.

FlowMsg is the carrier of information transferred between nodes on the FlowGraph. It is used to process the input and output of FlowFunc. FlowMsg uses the Mbuf to store information.

Context

If a large number of UDF processes exist and each UDF process may allocate a large amount of small memory, the Mbuf multi-memory pool is recommended. The following is an example.

Assume that the total size of the Mbuf pool is 10 GB. If the Mbuf multi-memory pool is not used, contiguous large memory cannot be allocated due to memory fragments. The details are as follows.

At time 1, multiple large memory blocks (3 GB) and small memory blocks (2 MB) are allocated.
At time 2, the large memory is no longer used and is deallocated to the memory pool to become free. At this time, the overall usage of the memory pool is as follows: 3 GB (free) + 2 MB (allocated) + 3 GB (free) + 2 MB (allocated) + 3 GB (free) + 2 MB (allocated). The available memory is over 9 GB.
At time 3, although the available memory is over 9 GB, 5 GB of contiguous memory fails to be allocated due to memory fragments (2 MB of memory blocks).

Examples

To resolve memory fragmentation in the above scenarios, extra memory pool configurations are required. The core solution is to enable Mbuf to support a configurable multi-pool architecture, where small memory blocks are allocated from a dedicated small memory pool.

The following figure shows an example of the Mbuf multi-memory pool.

At time 1, multiple large memory blocks (3 GB) and small memory blocks (2 MB) are allocated.
At time 2, the large memory is no longer used and is deallocated to the memory pool to become free. At this time, the overall usage of the memory pool is 3 GB (free) + 3 GB (free) + 3 GB (free) + 2 MB (allocated) + 2 MB (allocated). The available memory in memory pool 1 is over 9 GB, and the available memory in memory pool 2 is over 1 GB.

At time 3, memory pool 1 has over 9 GB of available contiguous memory, so a 5 GB memory allocation succeeds.

This solution is implemented by configuring ge.flowGraphMemMaxSize. For a configuration example, see example 2 in Table 2.

**Table 2** Examples
Example	Description	Code
Example 1	Allocate a memory pool with the size of 10 GB. The allocated memory size is not limited.	{"ge.flowGraphMemMaxSize", "10737418240"}
Example 2	Allocate two memory pools. The size of memory pool 1 is 10 GB, and the size of the allocated memory is not limited. The size of memory pool 2 is 2 GB, and the maximum size of the allocated memory is 2 MB. Allocations of 2 MB or less are preferentially served from memory pool 2; allocations larger than 2 MB are restricted to memory pool 1.	{"ge.flowGraphMemMaxSize", "10737418240,2147483648:2097152"}
Example 3	Allocate three memory pools. The size of memory pool 1 is 10 GB, and the size of the allocated memory is not limited. The size of memory pool 2 is 2 GB, and the maximum size of the allocated memory is 20 MB. The size of memory pool 3 is 1 GB, and the maximum size of the allocated memory is 2 MB. Memory allocations are routed to different pools based on the size of memory to be allocated. For instance, memory pool 3 preferentially handles allocations of up to 2 MB and exclusively handles allocations exceeding 20 MB; memory pool 2 preferentially handles allocations ranging from 2 MB to 20 MB.	{"ge.flowGraphMemMaxSize", "10737418240,2147483648:20971520,1073741824:2097152"}

ge.flowGraphMemMaxSize is described as follows:

Size of the memory that can be used by the network in FlowGraph scenarios. The value can be specified based on the network size. The unit is byte. Multiple memory pools can be configured. A maximum of 128 memory pools are supported. The configuration format is as follows: "Size of memory pool 1:Memory allocation limit,Size of memory pool 2:Memory allocation limit,...,Size of memory pool n:Memory allocation limit". If this option is not set, a single memory pool of 10 GB is used by default, and the size of memory that can be allocated is not limited.

Valid values for memory pool capacity are within the range [1024, 274877906944], in bytes. The combined capacity of all memory pools shall not exceed 256 GB, and the practical available capacity is subject to hardware constraints.

The memory allocation limit accepts a value of 0 or any integer in the range [1024, 2147483648], in bytes. This is an optional parameter with a default value of 0, which means no allocation size restriction is applied.

Parent topic: Appendices