[object Object]

[object Object][object Object]undefined[object Object]

• Description: This operator corresponds to the routing computation in the MoE model. It uses the computation result of the operator as the input and outputs the quantized routing matrix, [object Object], and other results for subsequent computation. This API has the following function changes based on . Select a proper API based on your actual requirements.

  • Added the drop mode. In this mode, the output is based on the number of tokens processed by each expert and equal to the value of·[object Object]. If the output exceeds the value, exceeded tokens are dropped. If the output falls short of the value, 0s are padded.
  • Added the optional output [object Object] in dropless mode. The output is based on the number of cumulative tokens (Cumsum) to be processed by each expert or the number of tokens (Count) to be processed by each expert.
  • Added the optional output [object Object] in drop mode. The output is based on the number of tokens to be processed by each expert before the drop operation.
  • Deleted the input [object Object].
  • Added the dynamic quantization mode.

• Formula:

  1. Flatten the input expertIdx with shape [NUM_ROWS, K] into a row for sorting, where NUM_ROWS indicates the number of input tokens and K indicates the number of experts selected for tokens. The sortedExpertIdx and corresponding sortedRowIdx are obtained.
  $$sortedExpertIdx, sortedRowIdx=keyValueSort(\text{flatten}(expertIdx))$$
  2. Use [object Object] for location mapping to obtain [object Object].
  $$expandedRowIdxOut[sortedRowIdx[i]]=i$$
  3. In dropless mode, collect statistics on the histogram of each expert in [object Object] and perform Cumsum to obtain [object Object].
  $$expertTokensCountOrCumsumOutOptional[i]=Cumsum(Histogram(sortedExpertIdx))$$
  4. In Drop mode, collect the histogram result of each expert in sortedExpertIdx to obtain expertTokensBeforeCapacityOutOptional.
  $$expertTokensBeforeCapacityOutOptional[i]=Histogram(sortedExpertIdx)$$
  5. Obtain the quantization result.

  • Static quantization:

    $$quantResult = round((x * scaleOptional) + offsetOptional)$$

  • Dynamic quantization:

    • If [object Object] is not specified:

      $$dynamicQuantScaleOutOptional = row\_max(abs(x)) / 127$$ $$quantResult = round(x / dynamicQuantScaleOutOptional)$$

    • If [object Object] is specified:

      $$dynamicQuantScaleOutOptional = row\_max(abs(x * scaleOptional)) / 127$$ $$quantResult = round(x * scaleOptional / dynamicQuantScaleOutOptional)$$
  6. Obtain expandedXOut based on quantResult.
  $$expandedXOut[expandedRowIdxOut[i]]=quantResult[i // K]$$

[object Object]

Each operator has calls. First, [object Object] is called to obtain the workspace size required for computation and the executor that contains the operator computation process. Then, [object Object] is called to perform computation.

[object Object]

• Parameters:

  [object Object]
  • Ascend 950PR/Ascend 950DT: The output expandedXOut data type supports only INT8.

• Returns:

  aclnnStatus: status code. For details, see [object Object]aclnn Return Codes[object Object].

  The first-phase API implements input parameter verification. The following errors may be thrown.

  [object Object]

[object Object]

• Parameters:

  [object Object]

• Returns:

  [object Object]: status code. For details, see .

[object Object]

• Deterministic computation:
  • [object Object] defaults to deterministic implementation.

[object Object]

The following example is for reference only. For details, see .