Constraints

Only HCCS interconnection between devices is supported. You can run the following command to query the interconnection information between cards:
1
npu-smi info -t topo -i 0
Use transWeight to configure the input x/weight matrix dimensions to meet the dimension relationship of matrix multiplication.

rank, rankSize, and rankRoot must meet the following conditions:
- 0 ≤ rank < rankSize
- 0 ≤ rankRoot < rankSize

If there are multiple users, ATB_SHARE_MEMORY_NAME_SUFFIX (see ATB Environment Variables) needs to be used to distinguish the shared memory and synchronize initialization information.
When the communication operator of the ATB exits abnormally, run the following commands to clear the residual data to avoid affecting subsequent operations.
```
rm -rf /dev/shm/sem.lccl*
rm -rf /dev/shm/sem.hccl*
ipcrm -a
```

When the AllGatherMatmulReduceScatter operator is called (that is, type of LinearParallelParam is set to ALL_GATHER_LINEAR_REDUCE_SCATTER), backend must be set to "lcoc", keepIntermediate must be set to false, and agDim, rsDim, and innerDimIsAg in twoDimTPInfo can be set. In addition, the requirement of agDim × rsDim = rankSize must be met. Quantization-related parameters are not supported. The value ranges of input dimensions m, k, and n are [1, 200000], [1, 32768], and [1, 32768], respectively.
agDim and rsDim must be powers of 2 and their minimum value is 2. The minimum value of rankSize is 4 and its maximum value is 16.
The constraints on the AlltoallvcAllGatherGroupMatmul and GroupMatmulReduceScatterAlltoallvc operators are as follows:
- When type is set to ALLTOALLVC_ALL_GATHER_GMM or GMM_REDUCE_SCATTER_ALLTOALLVC, the value of localExpertNums ranges from 1 to 16. ep×tp = rankSize.
- When type is set to ALLTOALLVC_ALL_GATHER_GMM or GMM_REDUCE_SCATTER_ALLTOALLVC, quantType can only be QUANT_TYPE_PER_CHANNEL or QUANT_TYPE_PER_TOKEN.
- Because AllToAllV uses variable-size communication, the output matrix of each card is different. m' is the maximum space for the output to ensure that the output of each card can be accommodated. When m' is insufficient for accommodating the output of the card, the token is truncated in the operator. To ensure that the staged execution scenario is not triggered, the value of m' can be m × rankSize × localExpertNum. The AlltoallvcAllGatherGroupMatmul and GroupMatmulReduceScatterAlltoallvc operators must meet the requirement of m' × 256 × sizeof(inputType) < 200 × 1024 × 1024/2.
In the Atlas 200T A2 Box16 heterogeneous subrack scenario, the communicator can be created only among the first eight or last eight cards. A communicator that contains both the first eight and last eight cards is not supported.
In the lcoc scenario, the maximum values of M, K, and N are 10000000, 100000, and 32768, respectively.
In the mc2 scenario, the value ranges of M, K, and N are [2, 10000000], [256, 65535), and [1, 32768], respectively.

Parent topic: LinearParallelOperation