Initialize

Ascend 950PR / Ascend 950DT

√

Atlas A3 training products / Atlas A3 inference products

√

Atlas A2 training products / Atlas A2 inference products

√

Atlas 200I/500 A2 inference products

x

Atlas inference products

x

Atlas training products

x

local_engine

Input

HIXL identifier, which must be unique among all participants in the link establishment.

• If an IPv4 address is used, the format can be host_ip:host_port or host_ip.
• If an IPv6 address is used, the format can be [host_ip]:host_port or [host_ip].

You are advised not to set this parameter to a loopback IP address, as it may lead to conflicts in scenarios involving multiple HIXL instances.

If host_port is set and its value is greater than 0, the current HIXL instance functions as the server and needs to listen on the specified port. If host_port is not set or its value is less than or equal to 0, the current HIXL instance functions as the client and will not start listening.

options

Input

Initialization parameter values. For details, see the following table.

OPTION_ENABLE_USE_FABRIC_MEM

Optional

String value "EnableUseFabricMem".

• 0: The Fabric Mem mode is disabled.
• 1: The Fabric Mem mode is enabled.

This parameter applies to scenarios where HCCS is used for D2RH and RH2D transmission.

Note: In cluster scenarios, this parameter must be set to the same value on all nodes. This parameter cannot be set together with "OPTION_BUFFER_POOL".

It is supported only by the Atlas A3 training products / Atlas A3 inference products .

OPTION_BUFFER_POOL

Optional

String value "BufferPool".

In scenarios where intermediate buffer is required for transmission:

• The size of RDMA-registered host memory is limited.
• Multiple small memory blocks (for example, 128 KB) need to be transmitted through relay transmission for performance optimization.

This parameter can be used to configure the size of the relay memory pool. The value format is "$BUFFER_NUM:$BUFFER_SIZE". By default, the system sets this parameter to "4:8" (unit: MB). You can set this parameter to 0:0 to disable the relay memory pool. In scenarios with concurrency, you are advised to increase the value of $BUFFER_NUM. In addition, the same value must be configured in all places where this parameter is used. For restrictions on the relay transmission mode, see the TransferSync API.

For Atlas A3 training products / Atlas A3 inference products , this parameter cannot be set together with "OPTION_ENABLE_USE_FABRIC_MEM".

Note: If this parameter is not set, the following restrictions apply:

• Atlas A2 training products / Atlas A2 inference products : For Atlas A2 training products / Atlas A2 inference products , only the Atlas 800I A2 inference server and A200I A2 Box heterogeneous subrack are supported. When the HCCS transmission protocol is used in the server, only D2D transmission is supported.

OPTION_RDMA_TRAFFIC_CLASS

Optional

String value "RdmaTrafficClass".

This parameter specifies the traffic class of the RDMA NIC. If both this parameter and the environment variable HCCL_RDMA_TC are configured, this parameter takes precedence. If only one of them is configured, the configured one takes effect.

Its value ranges from 0 to 255 and must be an integral multiple of 4. The default value is 132.

For more information, see Environment Variables.

OPTION_RDMA_SERVICE_LEVEL

Optional

String value "RdmaServiceLevel".

This parameter specifies the service level of the RDMA NIC. It has the same functionality as the environment variable HCCL_RDMA_SL. If both are configured, this parameter takes precedence. If only one of them is configured, the configured one takes effect.

The value range is [0, 7]. The default value is 4.

For more information, see Environment Variables.

OPTION_GLOBAL_RESOURCE_CONFIG

Optional

String value "GlobalResourceConfig", which is used to enable and configure global resources.

For the Fabric Mem mode (supported only by Atlas A3 training products / Atlas A3 inference products ), the configuration example is as follows:

{
    "fabric_memory.max_capacity": "128", //Size of the virtual memory pool. Value range: an integer in the range (0, 1024]. Default value: 32. Unit: TB.
    "fabric_memory.task_stream_num": "1", //Number of streams used by a single task in Fabric Mem mode. Value range: an integer in the range [1, 8]. Default value: 4.
}

The following is an example of setting parameters for the asynchronous link establishment/disconnection mechanism:

{
    "connect_pool.thread_num":"2", // Number of threads in the connection pool. Value range: an integer in the range [1, 64]. Default value: 2.
    "connect_pool.task_queue_capacity":"256" // Capacity of the task queue in the connection pool. Value range: an integer in the range [1, 65535]. Default value: 128.
}

The default listening port of the NIC on the device is 16666. If multiple processes use the same NIC, you can perform the following configuration:

{
    "comm_resource_config.listen_port": "26666" //Optional. Value range: an integer in the range [1, 65535]. If this parameter is not set, the automatically generated rank table does not contain the device_port field.
}

The following is an example of setting parameters for the link pool mechanism:

{
    "channel_pool.max_channel": "10", //Maximum number of links. Value range: an integer in the range (0, 512]. Default value: 512.
    "channel_pool.high_waterline": "0.3", // High watermark threshold for link recycle. Value range: a decimal number in the range (0, 1). This parameter must be configured together with channel_pool.low_waterline.
    "channel_pool.low_waterline": "0.1" // Low watermark threshold for link recycle, that is, the proportion of links to be retained after recycling. Value range: a decimal number in the range (0, 1). The value must be less than the high watermark threshold and must be configured together with the high watermark threshold.
}

When the link pool is working and the number of links reaches the high watermark threshold, the system selects a number of links (calculated as follows: number of current links – number of links corresponding to the low watermark threshold) to destroy (links that are transmitting data are not destroyed). The related parameters are calculated as follows:

• Number of links corresponding to the high watermark = max(1, floor(channel_pool.max_channel × channel_pool.high_waterline))
• Number of links corresponding to the low watermark = max(1, floor(channel_pool.max_channel × channel_pool.low_waterline))

In the example above, the high and low watermarks evaluate to 3 and 1, respectively. Before establishing a new link, the system checks whether the number of current links has reached 3. If so, it selects a number of links (calculated as follows: number of current links – 1) to destroy (links that are transmitting data are not destroyed).

When enabling the link pool mechanism, pay attention to the following:

• OPTION_GLOBAL_RESOURCE_CONFIG must be configured for all HIXL Engines in the cluster.
• When the TransferSync or TransferAsync API is called, the link pool automatically establishes a link if no available link exists.
• The link pool mechanism introduces additional overhead for transmission and link establishment, which may lead to performance deterioration.

OPTION_AUTO_CONNECT

Optional

String value "AutoConnect".

• 0: The Auto Connect mode is disabled.
• 1: The Auto Connect mode is enabled.

Note:

• If this parameter is enabled, link establishment can be skipped and transmission can be performed directly.
• If this parameter is enabled, abnormal links are automatically cleared when a transmission exception occurs or the peer end is destroyed. (The peer end destruction needs to be detected by the heartbeat mechanism. The default heartbeat interval is 10s.)

OPTION_LOCAL_COMM_RES

Optional

Local communication resource information, formatted as a JSON string.

• If this parameter is not set or is set to an empty string, related information is automatically generated. The collective communicator is used to establish links. The maximum number of links on a single card is 512.
• If version is set to "1.0" or "1.2" in the rank table, the communicator for collective communication is used to establish links. The maximum number of links on a single card is 512. You only need to configure the device information used by the current LLM-DataDist in the rank table. You do not need to configure the server_count and rank_id fields in the rank table. For details about the rank table, see the HCCL User Guide.
• If version is set to "1.3" in the rank table (recommended, the HDK version must be 25.5.0 or later and the Toolkit version must be 9.1.0 or later), the HixlCS capability is used to establish links and there is no limit on the maximum number of links. For details about the configuration format, see the communication resource field description. You only need to configure the version field. Other fields are automatically generated.

OPTION_LOCAL_COMM_RES

Mandatory

Local communication resource information, formatted as a JSON string. For details about the configuration format, see Table 3. If this option is left empty, related information will not be automatically generated.

UB:

{
  "version": "1.3",
  "net_instance_id": "superpod1_1",
  "endpoint_list": [
    {
      "protocol": "ub_ctp",
      "comm_id": "00000000007f020000100000df149001",
      "placement": "host",
      "dst_eid": "00000000007f030000100000df141c01"
    }
  ]
}

RoCE:

{
  "version": "1.3",
  "net_instance_id": "superpod1_1",
  "endpoint_list": [
    {
      "protocol": "roce",
      "comm_id": "192.168.100.100",
      "placement": "host"
    }
  ]
}

UBoE:

{
  "version": "1.3",
  "net_instance_id": "superpod1_1",
  "endpoint_list": [
    {
      "protocol": "uboe",
      "comm_id": "192.168.100.123",
      "placement": "device"
    }
  ]
}

UBG:

{
  "version": "1.3",
  "net_instance_id": "superpod_1",
  "endpoint_list": [
    {
      "protocol": "ubg",
      "comm_id": "0000000000ff0a80000000000a140200",
      "placement": "device"
    }
  ]
}

Note: The values in the preceding configuration example are for reference only. In actual use, you must query the actual communication resource configuration information in the current environment and replace the values accordingly. If you directly copy the example values, the communication will fail.

OPTION_AUTO_CONNECT

Optional

String value "AutoConnect".

• 0: The Auto Connect mode is disabled.
• 1: The Auto Connect mode is enabled.

Note:

• If this parameter is enabled, link establishment can be skipped and transmission can be performed directly.
• If this parameter is enabled, abnormal links are automatically cleared when a transmission exception occurs or the peer end is destroyed. (The peer end destruction needs to be detected by the heartbeat mechanism. The default heartbeat interval is 10s.)

version

String

Mandatory

Version number.

"1.3". This value requires the HDK version to be 25.5.0 or later and the Toolkit version to be 9.1.0 or later.

net_instance_id

String

Mandatory

Unique ID of the current SuperPoD.

The ID is unique for each SuperPoD.

endpoint_list

Array

Mandatory

List of available communication devices.

-

endpoint_list[].protocol

String

Mandatory

Communication protocol.

"roce"/"ub_ctp"/"ub_tp"/"uboe"

endpoint_list[].comm_id

String

Mandatory

Communication identifier.

• Set this parameter to ${eid} when protocol is set to ub_ctp or ub_tp.
• Set this parameter to the IPv4 or IPv6 NIC address when protocol is set to roce.
• Set this parameter to the IP address of the device UBoE NIC when protocol is set to uboe.

endpoint_list[].placement

String

Mandatory

Communication device location.

"host"/"device"

endpoint_list[].plane

String

Optional

Communication device plane.

This parameter is mandatory when the device is plane-specific and protocol is set to ub_ctp or ub_tp. The value must be unique for each plane, for example, "plane-a" or "plane-b".

endpoint_list[].dst_eid

String

Optional

${eid} of the peer communication device connected to the current communication device.

If protocol is set to ub_ctp and the peer device is directly connected in full-mesh mode, set this parameter to the peer ${eid}.

comm_resource_config.protocol_desc

String array

Optional

Communication protocol and communication device location configuration.

Currently, only ["uboe:device"] is supported, indicating the use of the UBOE protocol with the communication device located on the device. If OPTION_LOCAL_COMM_RES is not configured or if the endpoint_list within OPTION_LOCAL_COMM_RES is empty, the UBoE endpoint information will be generated automatically; otherwise, the configuration will be ignored.