[object Object]

[object Object][object Object]undefined[object Object]

• Description: Performs permute computation of MoE. It broadcasts tokens and optional probs based on the indices, sorts them, and slices them based on the range in rangeOptional.
• Formula:

  • When paddedMode is false, the formula is as follows: topK indicates the number of experts selected for each token. If Indices is two-dimensional, topK is equal to the size of the last dimension of Indices. If Indices is one-dimensional, topK is 1.

    $$sortedIndicesFirst=argSort(\text{flatten}(Indices))$$ $$sortedIndicesOut=argSort(sortedIndicesFirst)$$

    When rangeOptional[0] <= sortedIndicesOut[i] < rangeOptional[1]:

    $$permuteTokensOut[sortedIndicesOut[i] - rangeOptional[1]]=tokens[i//topK]$$

  • When paddedMode is true (not supported currently):

    $$permuteTokensOut[i]=tokens[Indices[i]]$$ $$sortedIndicesOut=Indices$$

[object Object]

Each operator has calls. First, [object Object] is called to obtain the workspace size required for computation and the executor that contains the operator computation process. Then, [object Object] is called to perform computation.

[object Object]

• Parameters:

  [object Object]

• Returns:

  [object Object]: status code. For details, see .

  The first-phase API implements input parameter verification. The following errors may be thrown.

  [object Object]

[object Object]

• Parameters:

  [object Object]

• Returns:

  [object Object]: status code. For details, see .

[object Object]

• Deterministic computation:
  • [object Object] defaults to deterministic implementation.

[object Object]

The following example is for reference only. For details, see .