[object Object][object Object][object Object]undefined[object Object]

• API function: Implements the fusion of AlltoAllv, Permute, and GroupedMatMul for routed experts, and achieves parallel fusion with the MatMul for shared experts. Communication is performed before computation..

• Formula:

  • Routed experts:
  $$ataOut = AlltoAllv(gmmX) \\ permuteOut = Permute(ataOut) \\ gmmY = permuteOut \times gmmWeight$$
  • Shared experts:
  $$mmY = mmX \times mmWeight$$

[object Object]

Each operator has calls. First, [object Object] is called to obtain the input parameters and compute the required workspace size based on the process. Then, [object Object] is called to perform computation.

[object Object]

• Parameters

  [object Object]

• Returns

  aclnnStatus: status code. For details, see .

  The first-phase API implements input parameter verification. The following errors may be thrown.

  [object Object]

[object Object]

• Parameters

  [object Object]

• Returns

  aclnnStatus status code. For details, see .

[object Object]

• Deterministic computing:

  • [object Object] defaults to a deterministic implementation.

• Shape variables used in parameter descriptions:

  • BSK: Number of tokens sent by the local rank, which is the sum of the sendCounts parameters. The value range is (0, 52428800).
  • H1: Hidden layer size of the routed experts. The value range is (0, 65536).
  • H2: Hidden layer size of the shared experts. The value range is (0, 12288].
  • e: Number of experts on a single rank. e <= 32. The maximum supported value for e * [object Object] is 256.
  • N1: head_num for routed experts. The value range is (0, 65536).
  • N2: head_num for shared experts. The value range is (0, 65536).
  • BS: Batch sequence size.
  • K: Number of experts selected via Top-K. The value range of K is [2, 8].
  • A: Number of tokens received by the local rank, which is the sum of the recvCounts parameters.
  • The sum of the A parameters across all ranks in the EP communication domain equals the sum of the BSK parameters across all ranks.

• [object Object]Atlas A3 training products/Atlas A3 inference products[object Object]: The communication volume per rank must be greater than or equal to 2 MB.

[object Object]

The following example is for reference only. For details, see .

• [object Object]Atlas A3 training products/Atlas A3 inference products[object Object]:

  [object Object]