[object Object]

[object Object][object Object]undefined[object Object]

• Description: Processes and combines the MoE FFN output during MoE computation.
• Formula:$$expertid=expertIdx[i,k]$$ $$out(i,j)=x1_{i,j}+x2_{i,j}+\sum_{k=0}^{K}(scales_{i,k}*(expandedX_{expandedRowIdx_{i+k*num\_rows},j}+bias_{expertid,j}))$$

[object Object]

Each operator has calls. First, [object Object] is called to obtain the workspace size required for computation and the executor that contains the operator computation process. Then, [object Object] is called to perform computation.

[object Object]

• Parameters:

  [object Object]
  • [object Object]Atlas A2 training series products/Atlas A2 inference series products[object Object] and [object Object]Atlas A3 training series products/Atlas A3 inference series products[object Object]:
    • [object Object] must be a 2D or 3D tensor. The supported data types are FLOAT16, BFLOAT16, and FLOAT32. The drop less and drop pad scenarios are supported.
    • [object Object]: In mixed precision mode, if [object Object] is BFLOAT16, [object Object] can be FLOAT32. In non-mixed precision mode, the data type must be the same as that of [object Object].
  • Ascend 950PR/Ascend 950DT:
    • [object Object] must be a 2D or 3D tensor. The supported data types are FLOAT16, BFLOAT16, and FLOAT32. The drop less and drop pad scenarios are supported.
    • The data type of scalesOptional can be different from that of expandedX.
  • [object Object]Atlas inference products[object Object]:
    • [object Object] must be a 2D tensor of type FLOAT16 or FLOAT32. The shape of expandedX must be 32-pixel aligned along the last axis (H).
    • [object Object], x2Optional[object Object]biasOptional[object Object]expertIdxOptional` support only null pointers.
    • Only 2 can be passed to [object Object].
    • The data type of [object Object] can be FLOAT16 or FLOAT32, and it must be the same as that of [object Object].

• Returns:

  [object Object]: status code. For details, see .

  The first-phase API implements input parameter verification. The following errors may be thrown:

  [object Object]

[object Object]

• Parameters:

  [object Object]

• Returns

  [object Object]: status code. For details, see .

[object Object]

1. Deterministic computing:

   • [object Object] defaults to a deterministic implementation.

2. [object Object]: number of rows.

   • [object Object]: number of experts selected from the total experts E
   • [object Object]: hidden size, that is, the length of each token sequence, which is the number of columns.
   • [object Object]: expert num, indicating the number of experts. E must be greater than or equal to [object Object].
   • [object Object]: expert capacity, that is, the threshold of the number of tokens that can be processed by an expert.

3. [object Object]: When [object Object] is set to 0 or 2, the value range of the tensor is [0, NUM_ROWS × K – 1]. When [object Object] is set to 1 or 3, the value range of the tensor is [–1, E × C – 1].

4. [object Object] must be specified before [object Object] is configured.

5. If [object Object] does not exist, [object Object] is 1.

6. If [object Object] exists, [object Object] must also exist.

7. The values and meanings of [object Object] are as follows:

   • 0: In the dropless scenario, [object Object] is arranged by column (corresponding to the output format of ).
   • 1: In the drop and pad scenario, [object Object] is arranged by column (corresponding to the output format of ).
   • 2: In the dropless scenario, [object Object] is arranged by row (corresponding to the output format of ).
   • 3: In the drop and pad scenario, [object Object] is arranged by row (corresponding to the output format of ).

[object Object]

The following example is for reference only. For details, see .