[object Object][object Object][object Object]undefined[object Object]

• Description: Performs routing on the computation result of , and then quantizes the result.
• Formula:

$$expandedExpertIdx,sortedRowIdx=keyValueSort(expertIdx,rowIdx)$$ $$expandedRowIdx[sortedRowIdx[i]]=i$$ $$expandedX[i]=quant[x[sortedRowIdx[i]\%numRows]]$$ [object Object]

Each operator has calls. First, [object Object] is called to obtain the workspace size required for computation and the executor that contains the operator computation process. Then, [object Object] is called to perform computation.

• [object Object]
• [object Object]

[object Object]

• Parameters:

  • [object Object] (aclTensor*, computation input): input of MoE, that is, token feature input. The input must be a 2D tensor with shape [object Object]. The data type can be FLOAT16, BFLOAT16, or FLOAT32. The must be ND. are supported.
  • [object Object] (aclTensor*, computation input): original row position corresponding to each position. The shape must be the same as that of [object Object]. The value starts from 0 and increases along dimension 1. The data type can be INT32. The must be ND. are supported.
  • [object Object] (aclTensor*, computation input): [object Object] experts corresponding to features in each row of the output of . This tensor must be 2D with shape (NUM_ROWS, K). The data type can be INT32. The must be ND. are supported.
  • [object Object] (int64_t, computation input): maximum number of rows to be processed, which is greater than or equal to 0. Only this many rows in [object Object] are valid.
  • [object Object] (double, computation input): required for quantization computation. The typical value is $S=\frac{R_{\max }-R_{\min }}{Q_{\max }-Q_{\min }}$, where $R_{max}$ indicates the maximum floating-point number, $R_{min}$ indicates the minimum floating-point number, $Q_{max}$ indicates the maximum fixed-point number, and $Q_{min}$ indicates the minimum fixed-point number.
  • [object Object] (double, computation input): required for quantization computation.
  • [object Object] (aclTensor*, compute output): 2D tensor, indicating features extended based on [object Object]. The shape is (min(NUM_ROWS, activeNum) * k, H). The data type is INT8. The must be ND. are not supported.
  • [object Object] (aclTensor*, compute output): mapping between [object Object] and [object Object]. It must be a 1D tensor with shape (NUM_ROWS*K, ). The data type can be INT32. The must be ND. are not supported.
  • [object Object] (aclTensor*, compute output): sorted expertIdx output. The data type can be INT32. The must be ND. are not supported.
  • [object Object] (uint64_t*, output): size of the workspace to be allocated on the device.
  • [object Object] (aclOpExecutor**, output): operator executor, containing the operator computation process.

• Returns

  [object Object] status code. For details, see .

  [object Object]

[object Object]

• Parameters:

  • [object Object] (void*, input): address of the workspace to be allocated on the device.
  • [object Object] (uint64_t, input): size of the workspace to be allocated on the device, which is obtained by calling [object Object].
  • [object Object] (aclOpExecutor*, input): operator executor, containing the operator computation process.
  • [object Object] (aclrtStream, input): stream for executing the task.

• Returns:

  [object Object] status code. For details, see .

[object Object]

• Deterministic computing:
  • The default deterministic implementation of [object Object] is used.

[object Object]

The following example is for reference only. For details, see .