Parameters

transWeight

bool

true

Whether to transpose weights. The default value is true.

rank

int

0

Communication ID of the current card

rankSize

int

0

Number of communication cards

• When backend is lcoc or mc2, the value is a power of 2.
• When backend is mc2, the value can only be 2, 4, or 8.

rankRoot

int

0

Primary communication rank

hasResidual

bool

false

Whether to add residual. If this parameter is set to false, residual is not added. If this parameter is set to true, residual is added. By default, residual is not added. mc2 does not support hasResidual.

backend

std::string

"hccl"

Communication backend. The value can be hccl, lccl, lcoc, or mc2. The LCCL function will be deleted in later versions. You are advised to use the HCCL function.

For the Atlas inference products, backend can only be hccl.

mc2 is supported only by Atlas A2 training products/Atlas A2 inference products and Atlas A3 inference products/Atlas A3 training products.

hcclComm

HcclComm

nullptr

Pointer to the HCCL communicator.

By default, this parameter is left blank. The ATB is created by users. If the user wants to manage the communicator, the communicator pointer needs to be passed. The ATB uses the passed communicator pointer to execute the communication operator.

commMode

CommMode

COMM_MULTI_PROCESS

Communication mode. For details, see CommMode. In the scenario where hccl multi-thread is applied, a communicator can only be imported externally. The lcoc operator is not affected. Retain the default value. mc2 supports only COMM_MULTI_THREAD.

rankTableFile

std::string

-

Path of the configuration file for cluster information, which applies to single-node and multi-node communication scenarios. Currently, only the HCCL backend is supported.

For details, see .

type

ParallelType

LINEAR_ALL_REDUCE

Weight parallel type. This parameter is valid only when backend is set to lcoc or mc2. mc2 supports only LINEAR_REDUCE_SCATTER.

• UNDEFINED: undefined.
• LINEAR_ALL_REDUCE: linear + AllReduce. This is the default value.

  Deterministic computation is involved. You can enable or disable it by using the environment variable LCCL_DETERMINISTIC.

• LINEAR_REDUCE_SCATTER: linear + reduce_scatter

  Deterministic computation is involved. You can enable or disable it by using the environment variable LCCL_DETERMINISTIC.

• ALL_GATHER_LINEAR: AllGather + linear
• PURE_LINEAR: linear
• ALL_GATHER_LINEAR_REDUCE_SCATTER: AllGather + linear + reduce_scatter. Deterministic computing is not supported.
• ALLTOALLVC_ALL_GATHER_GMM: AllToAllvc + AllGather + GroupMatmul.
• GMM_REDUCE_SCATTER_ALLTOALLVC: GroupMatmul + ReduceScatter + AllToAllvc.
• MAX: maximum value of enumerated types

keepIntermediate

bool

false

Whether to return the intermediate result. This parameter is valid only when ParallelType is set to ALL_GATHER_LINEAR. In other scenarios, this parameter must be set to false.

quantType

QuantType

QUANT_TYPE_UNQUANT

Quantization type. This parameter is valid only when backend is set to lcoc.

• QUANT_TYPE_UNDEFINED: default value
• QUANT_TYPE_UNQUANT: default value.
• QUANT_TYPE_PER_TENSOR: quantization of the entire tensor
• QUANT_TYPE_PER_CHANNEL: quantization of each channel in the tensor.
• QUANT_TYPE_PER_GROUP: quantization of the tensor divided by quantGroupSize
• QUANT_TYPE_PER_TOKEN: quantization of each token in the tensor.
• QUANT_TYPE_MAX: maximum value of enumerated types

quantGroupSize

int32_t

0

If the quantization type is QUANT_TYPE_PER_GROUP, the value can be 2, 4, 8, 16, 32, or 64. For other quantization types, the value is 0.

outDataType

aclDataType

ACL_DT_UNDEFINED

• For floating-point linear, if outDataType is set to ACL_DT_UNDEFINED, the data type of the output tensor is the same as that of the input tensor.
• For quantization linear, if the data type of the output tensor is different from that of the input tensor, set outDataType to the expected data type of the output tensor, for example, ACL_FLOAT16 or ACL_BF16.

commDomain

std::string

-

Communicator name used by a communication device group when there are multiple communicators. When backend is set to lccl and commMode is multi-process, commDomain needs to be set to a value ranging from 0 to 65535. The memory size of the communicator can be set for performance optimization. The configuration format is commDomain:commDomain size (MB), for example, 0:400. If the configurations of multiple operators in the same communicator conflict, the configuration of the first executed operator in the communicator is used.

Restriction: The default value is 200. Currently, the value cannot be less than 200.

twoDimTPInfo

TwoDimTPInfo

-

AllGather_Matmul_ReduceScatter operator parameter.

moeInfo

MoeInfo

1

MoE scenario parameter. This parameter is valid only when type is set to ALLTOALLVC_ALL_GATHER_GMM or GMM_REDUCE_SCATTER_ALLTOALLVC.

rsv[52]

uint8_t

{0}

Reserved

agDim

uint16_t

0

Number of cards on the allGather axis. The card numbers on the x axis are non-consecutive.

rsDim

uint16_t

0

Number of cards on the reduceScatter axis. The card numbers on the y axis are consecutive.

innerDimIsAg

uint8_t

1

Whether the ranks of allGather communication are consecutive. The value 1 indicates true, and the value 0 indicates false.

rsv[3]

uint8_t

{0}

The value must be 8 bytes.

localExpertNums

int16_t

1

Number of experts processed by the current card. This parameter is valid only when type is set to ALLTOALLVC_ALL_GATHER_GMM or GMM_REDUCE_SCATTER_ALLTOALLVC.

epSize

int8_t

1

Size of the EP domain. This parameter is valid only when type is set to ALLTOALLVC_ALL_GATHER_GMM or GMM_REDUCE_SCATTER_ALLTOALLVC.

The following condition must be met: epSize × tpSize = rankSize.

tpSize

int8_t

1

Size of the TP domain. This parameter is valid only when type is set to ALLTOALLVC_ALL_GATHER_GMM or GMM_REDUCE_SCATTER_ALLTOALLVC, and this parameter can only be set to 1.