Initialize

Applicability

Product

Supported (√/x)

Atlas 350 Accelerator Card

x

Atlas A3 training product / Atlas A3 inference product

Atlas A2 training product / Atlas A2 inference product

Atlas 200I/500 A2 inference product

x

Atlas inference product

x

Atlas training product

x

Note: For Atlas A2 training product / Atlas A2 inference product , only the Atlas 800I A2 inference server and A200I A2 Box heterogeneous subrack are supported.

Function Description

Initializes an HIXL instance. This API must be called before other APIs are called.

Prototype

1
Status Initialize(const AscendString &local_engine, const std::map<AscendString, AscendString> &options)

Parameters

Parameter

Input/Output

Description

local_engine

Input

HIXL identifier, which must be unique among all participants in the link establishment.

  • If an IPv4 address is used, the format can be host_ip:host_port or host_ip.
  • If an IPv6 address is used, the format can be [host_ip]:host_port or [host_ip].

You are advised not to set this parameter to a loopback IP address, as it may lead to conflicts in scenarios involving multiple HIXL instances.

If host_port is set and its value is greater than 0, the current HIXL instance functions as the server and needs to listen on the specified port. If host_port is not set or its value is less than or equal to 0, the current HIXL instance functions as the client and will not start listening.

options

Input

Initialization parameter values. For details, see the following table.

Table 1 Parameters in options

Parameter

Optional/Mandatory

Description

OPTION_BUFFER_POOL

Optional

String value "BufferPool".

In scenarios where intermediate buffer is required for transmission:

  • Direct H2H transmission via the HCCS protocol is not supported.
  • The size of RDMA-registered host memory is limited.
  • Multiple small memory blocks (for example, 128 KB) need to be transmitted through relay transmission for performance optimization.

This parameter can be used to configure the size of the relay memory pool. The value format is "$BUFFER_NUM:$BUFFER_SIZE". By default, the system sets this parameter to "4:8" (unit: MB). You can set this parameter to 0:0 to disable the relay memory pool. In scenarios with concurrency, you are advised to increase the value of $BUFFER_NUM. In addition, the same value must be configured in all places where this parameter is used. For restrictions on the relay transmission mode, see the TransferSync API.

For Atlas A3 training product / Atlas A3 inference product , this parameter cannot be set together with "OPTION_ENABLE_USE_FABRIC_MEM".

Note: If this parameter is not set, the following restrictions apply:

  • Atlas A2 training product / Atlas A2 inference product : For Atlas A2 training product / Atlas A2 inference product , only the Atlas 800I A2 inference server and A200I A2 Box heterogeneous subrack are supported. When the HCCS transmission protocol is used in the server, only D2D transmission is supported.
  • Atlas A3 training product / Atlas A3 inference product : When the HCCS transmission protocol is used, the host memory cannot be used as the remote cache.

OPTION_RDMA_TRAFFIC_CLASS

Optional

String value "RdmaTrafficClass".

This parameter specifies the traffic class of the RDMA NIC. If both this parameter and the environment variable HCCL_RDMA_TC are configured, this parameter takes precedence. If only one of them is configured, the configured one takes effect.

Its value ranges from 0 to 255 and must be an integral multiple of 4. The default value is 132.

For more information, see Environment Variables.

OPTION_RDMA_SERVICE_LEVEL

Optional

String value "RdmaServiceLevel".

This parameter specifies the service level of the RDMA NIC. It has the same functionality as the environment variable HCCL_RDMA_SL. If both are configured, this parameter takes precedence. If only one of them is configured, the configured one takes effect.

The value range is [0, 7]. The default value is 4.

For more information, see Environment Variables.

OPTION_GLOBAL_RESOURCE_CONFIG

Optional

String value "GlobalResourceConfig", which is used to enable and configure global resources.

OPTION_ENABLE_USE_FABRIC_MEM

Optional

String value "EnableUseFabricMem".

  • 0: The Fabric Mem mode is disabled.
  • 1: The Fabric Mem mode is enabled.

This parameter applies to scenarios where HCCS is used for D2RH and RH2D transmission.

Note: In cluster scenarios, this parameter must be set to the same value on all nodes. This parameter cannot be set together with "OPTION_BUFFER_POOL".

It is supported only by the Atlas A3 training product / Atlas A3 inference product .

OPTION_AUTO_CONNECT

Optional

String value "AutoConnect".

  • 0: The Auto Connect mode is disabled.
  • 1: The Auto Connect mode is enabled.

Note:

  • If this parameter is enabled, link establishment can be skipped and transmission can be performed directly.
  • If this parameter is enabled, abnormal links are automatically cleared when a transmission exception occurs or the peer end is destroyed. (The peer end destruction needs to be detected by the heartbeat mechanism. The default heartbeat interval is 10s.)

OPTION_LOCAL_COMM_RES

Optional

Local communication resource information, formatted as a JSON string.

The configuration method is as follows:

Only the device information used by the current LLM-DataDist instance in ranktable needs to be configured; the server_count and rank_id fields are not required. For details about ranktable, see Huawei Collective Communication Library (HCCL). This parameter can be left unconfigured or set to an empty string. If left empty, the relevant information is automatically generated.

The configuration examples and usage restrictions of `OPTION_GLOBAL_RESOURCE_CONFIG` are as follows:

For the Fabric Mem mode (supported only by Atlas A3 training product / Atlas A3 inference product ), the configuration example is as follows:

{
    "fabric_memory.max_capacity": "128", //Size of the virtual memory pool. Value range: an integer in the range (0, 1024]. Default value: 64. Unit: TB.
    "fabric_memory.task_stream_num": "1", //Number of streams used by a single task in Fabric Mem mode. Value range: an integer in the range [1, 8]. Default value: 4.
}

The following is an example of setting parameters for the link pool mechanism:

{
    "channel_pool.max_channel": "10", //Maximum number of links. Value range: an integer in the range (0, 512]. Default value: 512.
    "channel_pool.high_waterline": "0.3", //High watermark for triggering link destruction. Value range: a decimal number in the range (0, 1). This parameter must be configured together with channel_pool.low_waterline.
    "channel_pool.low_waterline": "0.1" //Low watermark for triggering link destruction. Value range: a decimal number in the range (0, 1). The value must be less than the high watermark.
}

The channel pool determines destruction based on the current channel count. If the count reaches the high watermark, it destroys (Current count – Low watermark) channels, skipping any that are actively transmitting before establishing a new connection. The relevant parameters are calculated as follows:

  • Channels at high watermark = max(1,static_cast<int32_t> (channel_pool.max_channel * channel_pool.high_waterline))
  • Channels at low watermark = max(1,static_cast<int32_t> (channel_pool.max_channel * channel_pool.low_waterline))

In the example above, the high and low watermarks evaluate to 3 and 1, respectively. Before each new connection, the system checks if the HIXL channel count has reached 3. If so, it destroys (Current count – 1) idle channels before proceeding with the new connection.

Before enabling the link pool mechanism, pay attention to the following:

  • OPTION_GLOBAL_RESOURCE_CONFIG must be configured for all HIXL Engines in the cluster.
  • When the TransferSync or TransferAsync API is called, if no related link exists, a new link will be established.
  • This increases the extra overhead for transmission and link establishment, which may lead to performance deterioration.

Example

Click GitCode, select the matching version, and obtain the sample from the examples/cpp directory.

Returns

  • SUCCESS: Success.
  • PARAM_INVALID: Incorrect parameter.
  • Other values: Failure.

Exception Handling

None

Restrictions

  1. This API must be used in conjunction with Finalize. After successful initialization, Finalize must be called before any exit to ensure resource release. Otherwise, issues may arise due to unexpected resource release order.
  2. Before initialization, aclrtSetDevice in Application Development (C&C++) must be called.