[object Object]

[object Object][object Object]undefined[object Object]

• Description: For [object Object] processed by [object Object], accumulates them back to the original [object Object]. This operator retrieves the input data stored in [object Object] based on the subscripts stored in [object Object]. If [object Object] data exists, [object Object] is multiplied by [object Object]. Then, this operator computes the cumulative sum and outputs the computation result.
• Formulas:$$topK\_num= permutedTokens.size(0) // routingMapOptional.size(0)$$ $$numExperts = probs.size(1)$$ $$numTokens = probs.size(0)$$ $$capacity = sortedIndices.size(0) // numExperts$$ (1) When probs is not None and paddedMode is true:$$permuteProbs [i//capacity,sortedIndices[i]]=probs[i]$$ $$permutedTokens = permutedTokens * permuteProbs$$ $$unpermutedTokens= zeros(restoreShape, dtype=permutedTokens.dtype, device=permutedTokens.device)$$ $$permuteTokenId, outIndex= sortedIndices.sort(dim=-1)$$ $$unpermutedTokens[permuteTokenId[i]] += permutedTokens[outIndex[i]]$$ (2) When probs is not None and paddedMode is false (T is the transpose operation):$$permuteProbs = probs.T.maskedSelect(routingMap.T)$$ $$permutedTokens = permutedTokens * permuteProbs$$ $$unpermutedTokens= zeros(restoreShape, dtype=permutedTokens.dtype, device=permutedTokens.device)$$ $$unpermutedTokens[i//topK\_num] += permutedTokens[sortedIndices[i]]$$ (3) When probs is None and paddedMode is true:$$permuteTokenId, outIndex= sortedIndices.sort(dim=-1)$$ $$unpermutedTokens[permuteTokenId[i]] += permutedTokens[outIndex[i]]$$ (4) When probs is None and paddedMode is false:$$unpermutedTokens[i//topK\_num] += permutedTokens[sortedIndices[i]]$$

[object Object]

Each operator has calls. First, [object Object] is called to obtain the workspace size required for computation and the executor covering the operator computation process. Then, [object Object] is called to perform computation.

[object Object]

• Parameters

  [object Object]

• Returns:

  [object Object]: status code. For details, see .

  The first-phase API implements input parameter verification. The following errors may be thrown:

  [object Object]

[object Object]

• Parameters

  [object Object]

• Returns:

  [object Object]: status code. For details, see .

[object Object]

• Deterministic computation:

  • [object Object] defaults to deterministic implementation.

• When topK_num <= 512 and paddedMode is false, the number of 1s or true values in each row of routingMap is fixed and less than 512.

• The following scenarios will be intercepted in later versions. If a warning is displayed, you are advised to rectify the fault.

  • paddedMode is true and topK_num > experts_num.
  • paddedMode is true and capacity > tokens_num.
  • The data type or shape of routingMap does not meet the requirements.
  • The data format of the input tensor is not ND.

[object Object]

The following example is for reference only. For details, see .