Initialize

Applicable Products

Product	Supported (√/x)
Ascend 950PR/Ascend 950DT	√
Atlas A3 training products/Atlas A3 inference products	√
Atlas A2 training products/Atlas A2 inference products	√
Atlas 200I/500 A2 inference products	x
Atlas inference products	x
Atlas training products	x

Note: For Atlas A2 training products/Atlas A2 inference products, only the Atlas 800I A2 inference server and A200I A2 Box heterogeneous subrack are supported.

Function Description

Initializes the LLM-DataDist instance.

Prototype

Status Initialize(const std::map<AscendString, AscendString> &options)

Parameters

Parameter	Input/Output	Description
options	Input	Initialization options. For details, see Table 1.

**Table 1** Options
Option	Optional/Mandatory	Description
OPTION_LISTEN_IP_INFO	Optional	Listening IP address and port number for the LLM-DataDist server. If configured, the LLM-DataDist instance acts as a server; otherwise, it acts as a client. When the LLM-DataDist instance acts as a server, the host IP address and port number must be provided. Example: "192.168.1.1:26000". Multiple IP addresses and port numbers are not supported.
OPTION_DEVICE_ID	Mandatory	Device ID of the current process, for example, "0". Single-process multi-card scenarios are not supported.
OPTION_SYNC_CACHE_WAIT_TIME	Optional	Timeout interval of KV operations, in milliseconds. If the parameter is not configured, the interval 1000 ms is used. The related APIs are as follows: PullKvCache PullKvBlocks PushKvCache PushKvBlocks
OPTION_LOCAL_COMM_RES	Optional	Local communication resource information, formatted as a JSON string. The configuration method is as follows: Only the device information used by the current LLM-DataDist instance in the rank table needs to be configured; the server_count and rank_id fields are not required. For details about the rank table, see Huawei Collective Communication Library (HCCL). This parameter can be left unconfigured or set to an empty string. If left empty, the relevant information is automatically generated. This method applies to the following products: Atlas A2 training product/Atlas A2 inference product Atlas A3 training product/Atlas A3 inference product In the Ascend 950PR/Ascend 950DT scenario, refer to https://gitcode.com/cann/hixl/issues/38 for the configuration format. In addition, set OPTION_TRANSFER_BACKEND to hixl. This option is mandatory. If it is left empty, related information will not be automatically generated.
OPTION_TRANSFER_BACKEND	Optional	The transmission backend engine used by LLM-DataDist. Currently, the supported backend is "hixl". The method of using the "hixl" transmission backend is as follows: When initializing the parameter, you need to specify OPTION_LISTEN_IP_INFO. When the hixl transmission backend is used, each transmission end can function as both a client and a server. Before initiating transmission with the peer end, you need to call LinkLlmClusters to establish a link.

Returns

LLM_SUCCESS: Success.
LLM_PARAM_INVALID: Incorrect parameter.
Other values: Failure.

Exception Handling

None

Constraints

This API must be used together with Finalize. After successful initialization, Finalize must be called before any exit to ensure resource release. Otherwise, issues may arise due to unexpected resource release order.

Parent topic: LLM-DataDist APIs