Initialize

Applicability

Product	Supported (√/x)
Atlas 350 Accelerator Card	x
Atlas A3 training product/Atlas A3 inference product	√
Atlas A2 training product/Atlas A2 inference product	√
Atlas 200I/500 A2 inference product	x
Atlas inference product	x
Atlas training product	x

Note: For Atlas A2 training product/Atlas A2 inference product, only the Atlas 800I A2 inference server and A200I A2 Box heterogeneous subrack are supported.

Function Description

Initializes the LLM-DataDist instance.

Prototype

Status Initialize(const std::map<AscendString, AscendString> &options)

Parameters

Parameter	Input/Output	Description
options	Input	Initialization parameter values. For details, see Table 1.

**Table 1** Parameters in options
Parameter	Optional/Mandatory	Description
OPTION_LISTEN_IP_INFO	Optional	Listening IP address and port number for the LLM-DataDist server. If configured, the LLM-DataDist instance acts as a server; otherwise, it acts as a client. When the LLM-DataDist instance acts as a server, the host IP address and port number must be provided. Example: "192.168.1.1:26000". Multiple IP addresses and port numbers are not supported.
OPTION_DEVICE_ID	Mandatory	Device ID of the current process, for example, "0". Single-process multi-card scenarios are not supported.
OPTION_SYNC_CACHE_WAIT_TIME	Optional	Timeout interval of KV operations, in milliseconds. If the parameter is not configured, the interval 1000 ms is used. The related APIs are as follows: PullKvCache PullKvBlocks PushKvCache PushKvBlocks
OPTION_LOCAL_COMM_RES	Optional	Local communication resource information, formatted as a JSON string. The configuration method is as follows: Only the device information used by the current LLM-DataDist instance in ranktable needs to be configured; the server_count and rank_id fields are not required. For details about ranktable, see Huawei Collective Communication Library (HCCL). This parameter can be left unconfigured or set to an empty string. If left empty, the relevant information is automatically generated. This method applies to the following products: Atlas A2 training product/Atlas A2 inference product Atlas A3 training product/Atlas A3 inference product
OPTION_TRANSFER_BACKEND	Optional	The transmission backend engine used by LLM-DataDist. Currently, the supported backend is "hixl". The method of using the "hixl" transmission backend is as follows: When initializing the parameter, you need to specify OPTION_LISTEN_IP_INFO. When the hixl transmission backend is used, each transmission end can function as both a client and a server. Before initiating transmission with the peer end, you need to call LinkLlmClusters to establish a link.

Example

Click Gitee, download the sample package of the matching version based on the tag name, and obtain the sample from the cplusplus/level1_single_api/11_llm_data_dist directory.

Returns

LLM_SUCCESS: Success.
LLM_PARAM_INVALID: Incorrect parameter.
Other values: Failure.

Exception Handling

None

Restrictions

This API must be used in conjunction with Finalize. After successful initialization, Finalize must be called before any exit to ensure resource release. Otherwise, issues may arise due to unexpected resource release order.

Parent topic: LLM-DataDist APIs