Initialize

Applicability

Product	Supported (Yes/No)
Atlas 350 Accelerator Card	No
Atlas A3 training product/Atlas A3 inference product	Yes
Atlas A2 training product/Atlas A2 inference product	Yes
Atlas 200I/500 A2 inference product	No
Atlas inference product	No
Atlas training product	No

Note: For the Atlas A2 training product/Atlas A2 inference product, only the Atlas 800I A2 inference server and A200I A2 Box heterogeneous subrack are supported.

Function Description

Initializes the LLM-DataDist instance.

Prototype

Status Initialize(const std::map<AscendString, AscendString> &options)

Parameters

Parameter	Input/Output	Description
options	Input	Initialization options. For details, see Table 1.

**Table 1** Parameters in options
Parameter	Optional/Mandatory	Description
OPTION_LISTEN_IP_INFO	Mandatory when the role is Prompt	IP address and port number on the device. Example: "192.168.1.1:26000". In the single-process multi-device scenario, use semicolons (;) to separate multiple IP addresses.
OPTION_DEVICE_ID	Mandatory	Device ID of the current process, for example, 0. In the single-process multi-device scenario, use semicolons (;) to separate multiple device IDs.
OPTION_SYNC_CACHE_WAIT_TIME	Optional	Timeout interval of KV operations, in milliseconds. If the parameter is not configured, the default interval 1000 ms is used. The related APIs are as follows: AllocateCache DeallocateCache PullKvCache PullKvBlocks CopyKvCache CopyKvBlocks PushKvCache PushKvBlocks
OPTION_BUF_POOL_CFG	Optional	Memory pool size and level. Configure this option to optimize the memory allocation performance and utilization. The value is a JSON string. For details about the fields, see Table 2. Example: { "buf_cfg":[{"total_size":2097152,"blk_size":256,"max_buf_size":8192}], "buf_pool_size": 2147483648 }
OPTION_ENABLE_SET_ROLE	Optional	Whether smooth role switchover is supported. The options are as follows: 1: supported 0: not supported. If this option is not configured, the function is not supported by default. Related API: SetRole

**Table 2** OPTION_BUF_POOL_CFG configuration
Option	Optional/Mandatory	Description
buf_cfg	Optional	Memory pool level. For details, see Table 3.
buf_pool_size	Optional	Memory pool size, in bytes.

**Table 3** buf_cfg configuration
Option	Optional/Mandatory	Description
total_size	Mandatory	Size of the memory pool at the current level, in bytes. Note: The value of total_size must be a multiple of 2 MB and a multiple of blk_size. The maximum value is 0xFFFFFFFF.
blk_size	Mandatory	Minimum memory that can be allocated at a time at the current level, in bytes. Note: The value must be 2ⁿ within the range of (0 MB, 2 MB], and less than or equal to the value of max_buf_size.
max_buf_size	Mandatory	Maximum memory that can be allocated at a time at the current level, in bytes. Note: The value of max_buf_size must be less than that of total_size and values of max_buf_size must be strictly increasing across different levels.

Returns

LLM_SUCCESS: Success.
LLM_PARAM_INVALID: Incorrect parameter.
Other values: Failure.

Exception Handling

None

Constraints

This API must be used in conjunction with Finalize. After successful initialization, Finalize must be called before any exit to ensure resource release. Otherwise, issues may arise due to unexpected resource release order.

Parent topic: LlmDataDist APIs