HcclCommConfig

Function

Defines the configurations (including the buffer size, deterministic computing switch, and communicator name) of a communicator during initialization.

Prototype

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27

const uint32_t HCCL_COMM_CONFIG_INFO_BYTES = 24;
const uint32_t COMM_NAME_MAX_LENGTH = 128;
const uint32_t BUFFER_NAME_MAX_LENGTH = 128;
const uint32_t UDI_MAX_LENGTH = 128;
const uint32_t HCCL_COMM_ALGO_MAX_LENGTH = 1600;
const uint32_t HCCL_COMM_RETRY_ENABLE_MAX_LENGTH = 50;
const uint32_t HCCL_COMM_RETRY_PARAMS_MAX_LENGTH = 128;
typedef struct HcclCommConfigDef {
    char reserved[HCCL_COMM_CONFIG_INFO_BYTES];    /* Reserved field, which cannot be modified. */
    uint32_t hcclBufferSize;
    uint32_t hcclDeterministic;
    char hcclCommName[COMM_NAME_MAX_LENGTH];
    char hcclUdi[UDI_MAX_LENGTH];
    uint32_t hcclOpExpansionMode;
    uint32_t hcclRdmaTrafficClass;
    uint32_t hcclRdmaServiceLevel;
    uint32_t hcclWorldRankID;
    uint64_t hcclJobID;
    uint8_t aclGraphZeroCopyEnable;
    int32_t hcclExecTimeOut;
    char hcclAlgo[HCCL_COMM_ALGO_MAX_LENGTH];
    char hcclRetryEnable[HCCL_COMM_RETRY_ENABLE_MAX_LENGTH];
    char hcclRetryParams[HCCL_COMM_RETRY_PARAMS_MAX_LENGTH];
    char hcclBufferName[BUFFER_NAME_MAX_LENGTH];
    uint32_t hcclQos;
    uint64_t hcclSymWinMaxMemSizePerRank;
} HcclCommConfig;

Parameters

• hcclBufferSize: size of the buffer for shared data. The value must be greater than or equal to 1, in MB.
• hcclDeterministic: switch of deterministic computing.
  The following lists the values and meanings supported by different AI processors.

  • Atlas 350 Accelerator Card: 1 or left blank, indicating that deterministic computing is enabled for reduction communication operators AllReduce, ReduceScatter, Reduce, and ReduceScatterV.
  • Atlas A3 training product/Atlas A3 inference product:
    • 0 (default): Disables deterministic computing.
    • 1: Enables deterministic computing for reduction communication operators AllReduce and ReduceScatter.
    • 2: In single-operator mode, the value 2 functions the same as 1. The static graph mode does not support the value 2.
  • Atlas A2 training product/Atlas A2 inference product:
    • 0 (default): Disables deterministic computing.
    • 1: Enables deterministic computing for reduction communication operators AllReduce, ReduceScatter, Reduce, and ReduceScatterV.
    • 2: Enables strict deterministic computing for reduction communication operators, which provides the order-preservation feature (guaranteeing a consistent reduction order across all bits based on determinism). The value is applicable to communication operators AllReduce, ReduceScatter, and ReduceScatterV, subject to the following conditions:
      • Only the symmetric multi-server distribution scenario is supported.
      • When order preservation is enabled, the saturation mode is not supported, and only the INF/NaN mode is supported.
      • Compared with deterministic computing, enabling order preservation will cause performance deterioration. You are advised to use this function in inference scenarios.

  

  If deterministic computing is disabled, the results of multiple executions may be different. This is generally caused by asynchronous multi-thread executions during operator implementation, which changes the accumulation sequence of floating-point numbers. When deterministic computing is enabled, the same output is generated if an operator is executed for multiple times with the same hardware and input.

  By default, deterministic computing does not need to be enabled. However, if the model execution results are different or precision tuning is required, you can enable deterministic computing to assist debugging and optimization. Note that enabling deterministic computing will slow down operator execution and degrade performance.

• hcclCommName: communicator name, with a maximum of 128 characters.

  The communicator name must be unique. If not specified, the name is generated by HCCL.

• hcclUdi: user-defined information, with a maximum of 128 characters. By default, this parameter is left blank.
• hcclOpExpansionMode: expansion mode of the communication operators, configured at the communicator granularity.

  The following lists the values and meanings supported by different AI processors.

  Atlas 350 Accelerator Card:

  • 0: The adaptive expansion mode is used by default, which automatically selects the most appropriate mode based on network topology and data size.
  • 2: Communication operators are expanded on the AICPU compute unit.

    This value is applicable to Broadcast, Reduce, AllReduce, Scatter, ReduceScatter, AllGather, AlltoAll, AlltoAllV, and AlltoAllVC, Send, Recv, and BatchSendRecv.

    In graph (Ascend IR) or graph capture (ACLGraph) mode, when the communication algorithm uses the AICPU mode, the number of concurrent graphs on a single device cannot exceed 6. Otherwise, the communication may be blocked because the AICPU cores are fully occupied.

  • 3: Communication operators are expanded on the Vector Core compute unit on the device side. This value is not supported in the current version.
    • This option supports only the symmetric networking and inference features.
    • If the data size does not meet the running requirements of the Vector Core, some operators are automatically switched to the default mode.
    • This configuration only supports Broadcast, Reduce, AllReduce, ReduceScatter, Scatter, AllGather, AlltoAll, and AlltoAllV operators, and is limited to single-server scenarios only.
      • For the Broadcast, Scatter, AllGather, AlltoAll, and AlltoAllV operators, the supported data types include int8, uint8, int16, uint16, int32, uint32, int64, uint64, float16, float32, and bfp16.
      • For the Reduce, AllReduce, and ReduceScatter operators, the supported data types include int8, int16, int32, float16, float32, and bfp16.
    • Under this configuration, AllReduce, ReduceScatter, AllGather, and AlltoAll operators support core control capabilities. You are advised to configure the number of Vector Cores according to the concurrency of compute and communication operators in actual service scenarios.

      If the number of Vector Cores allocated during service compilation fails to meet the algorithm orchestration requirements, HCCL will throw an error and prompt the minimum number of Vector Cores required.

  • 4: Communication operators are expanded on the Vector Core compute unit on the device side and do not switch modes with data size change. If the running conditions of the Vector Core are not met, the process will terminate with an error. This value is not supported in the current version.
    • This option supports only the symmetric networking and inference features.
    • For the operators supported by this option and their constraints, see option 3.
  • 5: Communication operators are expanded on the collective communication unit (CCU) and uses the memory slice (MS) mode. This value is not supported in the current version.

    In MS mode, the CCU on-chip memory slice serves as a transit to communicate with multiple remote ends, saving the memory read/write bandwidth. The MS features small size and fast speed. When CCU resources are insufficient, the system automatically switches to the AICPU mode (2).

  • 6: Communication operators are expanded on the CCU and uses the scheduling mode.

    In scheduling mode, the CCU serves as a scheduler to dispatch UB WQE tasks to the UB engine. The scheduling mode does not use the CCU on-chip MS. Instead, it performs on-chip memory to on-chip memory data transfer between two ranks.

    For the AllReduce, ReduceScatter, and Reduce operators in single-server communication, when the data size exceeds a certain value, the system automatically switches to the AICPU mode (2) to prevent performance degradation. (The threshold is not fixed and may vary depending on factors such as the operator running mode and network scale.)

    When CCU resources are insufficient, the system automatically switches to the AICPU mode (2).

  Atlas A3 training product/Atlas A3 inference product:

  • 0: The default expansion mode is used. By default, the Atlas A3 training product/Atlas A3 inference product uses the AICPU compute unit on the device side.
  • 2: Communication operators are expanded on the AICPU compute unit.
  • 3: Communication operators are expanded on the Vector Core compute unit on the device side.
    • This option supports only the symmetric networking and inference features.
    • If the data size does not meet the running requirements of the Vector Core, some operators are automatically switched to the default mode.
    • This configuration option supports only the Broadcast, AllReduce, ReduceScatter, AllGather, AlltoAll, AlltoAllV, and AlltoAllVC operators.
      • For the Broadcast operator, supported data types include int8, uint8, int16, uint16, int32, uint32, float16, float32, and bfp16. It only supports single-server communication within a supernode and single-operator mode. Multi-server and inter-supernode communication are not supported.
      • For the AllReduce operator, the data type can be int8, int16, int32, float16, float32, or bfp16. The reduce operation type can only be sum, max, or min. Only single-server or multi-server communication within a supernode is supported. Communication between supernodes is not supported.
      • For the ReduceScatter operator, the data type can be int8, int16, int32, float16, float32, or bfp16. The reduce operation type can only be sum, max, or min. Only single-server or multi-server communication within a supernode is supported. Communication between supernodes is not supported.
      • For the AllGather, AlltoAll, AlltoAllV, and AlltoAllVC operators, the data type can be int8, uint8, int16, uint16, int32, uint32, float16, float32, or bfp16. Only single-server or multi-server communication within a supernode is supported. Communication between supernodes is not supported.
    • For the Broadcast, AllReduce, ReduceScatter, AllGather, and AlltoAll (single-server communication) operators, when the data size exceeds a certain value, the system automatically switches to the AICPU mode (2) to prevent performance degradation. (The threshold is not fixed and may vary depending on factors such as the operator running mode, whether deterministic computing is enabled, and network scale.) For the AlltoAllV, AlltoAllVC, and AlltoAll (multi-server communication) operators, the system does not automatically switch to the AICPU mode (2). To prevent performance degradation, you are advised to use the AIV mode 3 when the maximum communication data size between any two ranks does not exceed 1 MB. Otherwise, use the AICPU mode 2.
    • Under this configuration option, the collective communication supports the core control capability. You are advised to configure the number of vector cores based on the concurrency of compute operators and communication operators in actual application scenarios.
      • For the Broadcast operator, you are advised to allocate at least ranksize vector cores.
      • For the AllReduce, ReduceScatter, AllGather, AlltoAll, AlltoAllV, and AlltoAllVC operators, you are advised to allocate at least max(2, ranksize/20 + 1) vector cores.

      If the number of Vector Cores allocated during service compilation fails to meet the algorithm orchestration requirements, HCCL will throw an error and prompt the minimum number of Vector Cores required.

  • 4: Communication operators are expanded on the Vector Core compute unit on the device side and do not switch modes with data size change. If the running conditions of the Vector Core are not met, the process will terminate with an error.
    • This option supports only the symmetric networking and inference features.
    • The value is applicable the AllReduce, ReduceScatter, AllGather, AlltoAll, AlltoAllV, and AlltoAllVC operators. For the data types and constraints of related operators, see the option 3.
    • In this configuration, the collective communication supports the core control capability. The number of Vector Cores required by different operators is the same as that of the option 3.
  Atlas A2 training product/Atlas A2 inference product:

  • 0: The default expansion mode is used. By default, the Atlas A2 training product/Atlas A2 inference product uses the CPU on the host side.
  • 1: Communication operators are expanded on the CPU on the host side.
  • 2: Communication operators are expanded on the AICPU compute unit.

    This option supports only the AllGather, AlltoAll, AlltoAllV, and AlltoAllVC operators.

    In graph (Ascend IR) or graph capture (ACLGraph) mode, when the communication algorithm uses the AICPU mode, the number of concurrent graphs on a single device cannot exceed 6. Otherwise, the communication may be blocked because the AICPU cores are fully occupied.

  • 3: Communication operators are expanded on the Vector Core compute unit on the device side.
    • This option supports only the symmetric networking and inference features.
    • If the data size does not meet the running requirements of the Vector Core, some operators are automatically switched to the default mode.
    • This configuration option supports only the Broadcast, AllReduce, AlltoAll, AlltoAllV, AlltoAllVC, AllGather, ReduceScatter, AllGatherV, and ReduceScatterV operators.
      • For the Broadcast operator, the data type can be int8, uint8, int16, uint16, int32, uint32, float16, float32, or bfp16. Only the single-operator mode with eight or fewer devices in the single-server scenario is supported.
      • For the AllReduce operator, the data type can be int8, int16, int32, float16, float32, or bfp16. The reduce operation type can only be sum, max, or min.
      • For the AlltoAll, AlltoAllV, and AlltoAllVC operators, the data type can be int8, uint8, int16, uint16, int32, uint32, float16, float32, or bfp16. For the AlltoAllV and AlltoAllVC operators, only single-server scenarios are supported. The graph mode of the AlltoAll operator supports only single-server scenarios.
      • For the AllGather operator, the data type can be int8, uint8, int16, uint16, int32, uint32, float16, float32, or bfp16. The graph mode of this operator supports only single-server scenarios.
      • For the ReduceScatter operator, the data type can be int8, int16, int32, float16, float32, or bfp16. The reduce operation type can only be sum, max, or min. The graph mode of this operator supports only single-server scenarios.
      • For the AllGatherV operator, the data type can be int8, uint8, int16, uint16, int32, uint32, float16, float32, or bfp16. Only the single-operator mode is supported.
      • For the ReduceScatterV operator, the data type can be int8, int16, int32, float16, float32, or bfp16. The reduce operation type can only be sum, max, or min.
    • Under this configuration option, the collective communication supports the core control capability. You are advised to configure the number of vector cores based on the concurrency of compute operators and communication operators in actual application scenarios.
      • For the AllReduce, ReduceScatter, and ReduceScatterV operators, you are advised to allocate at least 24 cores.
      • For the Broadcast, AlltoAll, AlltoAllV, AlltoAllVC, AllGather, and AllGatherV operators, you are advised to allocate at least 16 cores.

      If the number of Vector Cores allocated during service compilation fails to meet the algorithm orchestration requirements, HCCL will throw an error and prompt the minimum number of Vector Cores required.

  • 4: Communication operators are expanded on the Vector Core compute unit on the device side and do not switch modes with data size change. If the running conditions of the Vector Core are not met, the process will terminate with an error.
    • This option supports only the symmetric networking and inference features.
    • The value is applicable only to the AllReduce, AlltoAll, AlltoAllV, AlltoAllVC, AllGather, and ReduceScatter operators. For the data types and constraints of related operators, see the option 3.
    • In this configuration, the collective communication supports the core control capability. The number of Vector Cores required by different operators is the same as that of the option 3.
  

  • If the expansion mode is set to 3 (AIV) or 4 (AIV Only), multi-communicator parallelism is not supported. Otherwise, unexpected behavior may occur.
  • For the Atlas A2 training product/Atlas A2 inference product, when the expansion mode is set to 3 or 4 and hcclDeterministic is set to 1 (to enable deterministic computing), deterministic computing takes effect only for the AllReduce and ReduceScatter operators with a data size of less than or equal to 8 MB in the single-operator and graph modes of a single server. In other scenarios and for other operators, the hcclDeterministic configuration is used.
  • For the Atlas A2 training product/Atlas A2 inference product, if hcclDeterministic is set to 2 (to enable order preservation), hcclOpExpansionMode cannot be set to 3 or 4, as the order preservation feature takes precedence.
  • For the Atlas A3 training product/Atlas A3 inference product, when the expansion mode is set to 3 or 4 and hcclDeterministic is set to 1 (to enable deterministic computing) or 2 (to enable order preservation), deterministic computing takes effect only for the AllReduce and ReduceScatter operators with a data size of less than 8 MB. In other scenarios and for other operators, the hcclDeterministic configuration is used.
• hcclRdmaTrafficClass: traffic class of the RDMA NIC. The value ranges from 0 to 255 and must be an integral multiple of 4.

  In the RoCE V2 protocol, this parameter corresponds to the Type of Service (ToS) field in the IP packet header. This parameter consists of 8 bits in total. Bits 0 and 1 are fixed at 0, and bits 2 to 7 represent the DSCP value (calculated by dividing this parameter value by 4).

  Note:

  0xFFFFFFFF is used as the priority judgment identifier. When this parameter is set to 0xFFFFFFFF, the communicator is invalid. The environment variable or the default value 132 is used based on the priority.

• hcclRdmaServiceLevel: service level (SL) of the RDMA NIC. The value must be the same as the PFC priority set for the NIC. Otherwise, performance may degrade.

  The value must be an unsigned integer ranging from 0 to 7.

  Note:

  0xFFFFFFFF is used as the priority judgment identifier. When this parameter is set to 0xFFFFFFFF, the communicator is invalid. The environment variable or the default value 4 is used based on the priority.

• hcclWorldRankID: This parameter is used in the Network Scale Load Balance-Data Plane (NSLB-DP) scenario and indicates the global rank ID of the current process in the AI framework (such as PyTorch).
• hcclJobID: This parameter is used in the NSLB-DP scenario and indicates the unique ID of the current distributed service, which is generated by the AI framework.
• aclGraphZeroCopyEnable: This parameter is valid only for Reduce operators in graph capture (ACLGraph) mode and is used to determine whether to enable the zero-copy function for these operators.
  • 0 (default): disables the zero-copy function.
  • 1: enables the zero-copy function.
• hcclExecTimeOut: During distributed training or inference, tasks executed by different device processes may be inconsistent, for example, only specific processes save the checkpoint data. This parameter controls the synchronization wait time during task execution between devices. Within this configured time, each device process waits for other devices to perform communication synchronization. The unit is second. For details about the value range and restrictions for different product models, see the environment variable HCCL_EXEC_TIMEOUT.

  Note:

  0xFFFFFFFF is used as the priority judgment identifier. When this parameter is set to 0xFFFFFFFF, the communicator is invalid. The environment variable or the default value 1836 is used based on the priority.

• hcclAlgo: configures the communication algorithms between servers and SuperPoDs for collective communication. The algorithms can be configured globally or by operator. Note that HCCL provides the adaptive algorithm selection function, so that it can select an appropriate algorithm by default based on the product form, data size, and server count, which usually requires no manual configuration. Enabling this parameter to specify the inter-server communication algorithm would invalidate the adaptive algorithm selection function.
  For details about the parameters and algorithm types supported by different product models, see the environment variable HCCL_ALGO. The configuration method is as follows:

  • Configuring the algorithm globally: hcclAlgo = "level0:NA;level1:<algo>;level2:<algo>". Example:

    hcclAlgo = "level0:NA;level1:H-D_R"

  • Configuring the algorithm by operator: hcclAlgo = "<op0>=level0:NA;level1:<algo0>;level2:<algo1>/<op1>=level0:NA;level1:<algo3>;level2:<algo4>". Example:

    # The AllReduce operator uses the Ring algorithm and the AllGather operator uses the RHD algorithm. Other operators automatically select a communication algorithm based on the product form, rank count, and data size.
    hcclAlgo = 
    "allreduce=level0:NA;level1:ring/allgather=level0:NA;level1:H-D_R"

• hcclRetryEnable: enables or disables the retry feature of the HCCL operator. If an SDMA or RDMA CQE error is reported during the execution of a communication operator, HCCL attempts to retry the communication operator. It is supported only by the Atlas A3 training product/Atlas A3 inference product.

  You can use this parameter to configure whether to enable the retry feature in the communicators of the inter-server and inter-SuperPoD physical layers. Each layer supports two states: enabled and disabled. For details about the constraints, see the environment variable HCCL_OP_RETRY_ENABLE. The configuration method is hcclRetryEnable = "L1:1, L2:0". The parameter values are as follows:

  • L1 indicates that the physical scope of the communicator is the communicator between servers. 0 (default value) indicates that the retry feature is disabled for inter-server communication tasks in the communicator, while 1 indicates that the retry feature is enabled for inter-server communication tasks in the communicator.
  • L2 indicates that the physical scope of the communicator is the communicator between SuperPoDs. 0 (default value) indicates that the retry feature is disabled for inter-SuperPoD communication tasks in the communicator, while 1 indicates that the retry feature is enabled for inter-SuperPoD communication tasks in the communicator.
• hcclRetryParams: configures the wait period for the first retry, the maximum number of retries, and the interval between two retries after the HCCL operator retry feature is enabled through the hcclRetryEnable parameter. It is supported only by the Atlas A3 training product/Atlas A3 inference product.
  For details about the constraints, see the environment variable HCCL_OP_RETRY_PARAMS. The configuration method is hcclRetryParams = "MaxCnt:3, HoldTime:5000, IntervalTime:1000". The parameter values are as follows:

  • MaxCnt: maximum retransmission attempts. The value is of the uint32 type. The value range is [1,10]. The default value is 1.
  • HoldTime: wait period from the time when a communication operator execution failure is detected to the time when the communication operator is retried for the first time. The value is of the uint32 type. The value range is [0,60000], with the default value of 5000, in millisecond.
  • IntervalTime: interval between two retries of the same communication operator. The value is of the uint32 type. The value range is [0,60000], with the default value of 1000, in millisecond.
• hcclBufferName: CCLBuffer name, with a maximum of 128 characters. Multiple communicators using the same buffer name share the same CCLBuffer. If this parameter is not specified, CCLBuffer is not shared. Note that communicators with the same CCLBuffer name must dispatch operators to the same stream.
• hcclQos: QoS level of the hyperplane. The value ranges from 0 to 7. The default value is 6.
• hcclSymWinMaxMemSizePerRank: size of the symmetric memory reserved for each rank in the current communicator, in GB. The value ranges from 1 to maximum physical memory that can be allocated in the environment. The default value is 16. Currently, this option can be used only on the Atlas A3 training product/Atlas A3 inference product.

Configuration Priority