[object Object]

[object Object][object Object]undefined[object Object]

• Description: Performs sigmoid or softmax calculation on input x during MoE calculation, sorts the calculation results by group, and selects the top k experts based on the group sorting result.

• Formula:

  Step 1: Normalization

  Normalize the input x based on [object Object]:

  $$normOut = \begin{cases} SoftMax(x), & normType = 0 \\ Sigmoid(x), & normType = 1 \end{cases}$$

  Step 2: Add a bias.

  If bias is not empty, add the bias to obtain the value used for selection:

  $$normValue = normOut + bias$$

  Otherwise, $normValue = normOut$.

  Step 3: Group filtering (performed only when [object Object] > 1)

  Group [object Object] by [object Object] and calculate the score of each group based on [object Object]:

  $$groupedValue = Reshape(normValue,\ [batch,\ groupCount,\ -1])$$ $$groupScore = \begin{cases} ReduceMax(groupedValue,\ dim=-1), & groupSelectMode = 0 \\ ReduceSum(TopK(groupedValue,\ k=2,\ dim=-1),\ dim=-1), & groupSelectMode = 1 \end{cases}$$

  Select the [object Object] groups with the highest scores and set the corresponding positions of the unselected groups to $-\infty$:

  $$groupIdx = TopK(groupScore,\ k=kGroup).indices$$ $$normValue = Mask(groupedValue,\ groupIdx,\ fillValue=-\infty)$$

  Step 4: Top-K expert selection

  Obtain the expert index by taking the top K of [object Object]. Here, only [object Object] is required.

  $$y, expertIdxOut = TopK(normValue[groupIdx, :],\ k=k)$$

  Step 5: Renormalization and Scaling

  When [object Object] is 1, normalization is performed. When [object Object] is 0, the [object Object] parameter takes effect. When [object Object] is 1, renorm is performed.

  $$if\ (normType = 1)\ or\ (normType = 0\ and\ renorm = 1):$$ $$\quad yOut = \frac{normOut}{ReduceSum(normOut,\ dim=-1) + eps}$$

  Final output:

  $$yOut = yOut \times routedScalingFactor$$

  Step 6: Optional output

  If [object Object] is [object Object], the third output is [object Object]. Otherwise, the output is empty.

[object Object]

Each operator consists of . You must first call the [object Object] API to obtain the required workspace size and the executor that contains the operator computation flow, and then call the [object Object] API to execute the computation.

[object Object]

• Parameters:

  [object Object]

• Returns:

  [object Object]: status code. For details, see .

  The first-phase API implements input parameter validation. The following error codes may be returned.

  [object Object]

[object Object]

• Parameters:

  [object Object]

• Returns:

  [object Object]: status code. For details, see .

[object Object]

• Deterministic computation:
  • [object Object] defaults to a deterministic implementation.

• Input shape restrictions:
  • The last dimension (i.e., the number of experts) of [object Object] must be less than or equal to 2048.
• Input value range restrictions:
  • 1 ≤ [object Object] ≤ [object Object].
  • 1 ≤ [object Object] ≤ [object Object], and the value of [object Object] * [object Object] must be greater than or equal to k.
  • [object Object] > 0, [object Object] can be exactly divided by [object Object], and the result is greater than [object Object]. In addition, the result of multiplying the result of 32-number alignment by [object Object] is less than or equal to 2048.
  • renorm can only be 0 or 1, indicating that the norm operation is performed before the topk operation.
• Other restrictions:
  • The value of [object Object] can be 0 or 1. The value 0 indicates that the groups are sorted based on the maximum value, and the value 1 indicates that the groups are sorted based on the sum value of [object Object].
  • The value of [object Object] can be 0 or 1. The value 0 indicates that the softmax function is used, and the value 1 indicates that the sigmoid function is used.
  • The value of [object Object] can be [object Object] or [object Object]. The value [object Object] indicates that the output is enabled, and the value [object Object] indicates that the output is disabled.

[object Object]

The following example is for reference only. For details, see .