[object Object][object Object][object Object]undefined[object Object]

• Description: Applies sigmoid to input [object Object] during MoE computation, sorts the calculation results by group, and selects the top k experts based on the group sorting result.

• Formula:

  Computes sigmoid of the input element-wise.

  $$sigmoidRes=sigmoid(x)$$

  Add addNum:

  $$normOut = sigmoidRes + addNum$$

  Group the computation results based on [object Object], sort each group based on the sum value of top N, and select the first [object Object] groups.

  $$groupOut, groupId = TopK(ReduceSum(TopK(Split(normOut, groupCount), k=2, dim=-1), dim=-1),k=kGroup)$$

  Obtain the corresponding element in [object Object] based on the [object Object] obtained in the previous step, and perform topK on the data to obtain the indices result.

  $$normY,indices=TopK(normOut[groupId, :],k=k)$$

  Select [object Object] from [object Object] based on indices.

  $$y = gather(sigmoidRes, indices)$$

  If [object Object] is [object Object], [object Object] is computed based on the input scale parameter to obtain the result of [object Object].

  $$y = y / (ReduceSum(y, dim=-1)) × scale$$

  If [object Object] is [object Object], the physical experts in indices are mapped to logical experts based on the input [object Object] and [object Object] to obtain the output indices.

[object Object]

Each operator consists of . You must first call the [object Object] API to obtain the required workspace size and the executor that contains the operator computation flow, and then call the [object Object] API to execute the computation.

• [object Object]

• [object Object]

[object Object]

• Parameters:

  • [object Object] (aclTensor*, computation input): [object Object] on the device. Each token corresponds to the score of each expert. The shape is (numToken, expertNum). The data type can be FLOAT16, BFLOAT16, or FLOAT32. The must be ND. are supported.
  • [object Object] (aclTensor*, computation input): [object Object] on the device, which is the bias value for computation with input [object Object]. The shape is (expertNum). The data type must be the same as that of [object Object]. The must be ND. are supported.
  • [object Object] (aclTensor*, computation input): [object Object] on the device. This parameter is not enabled when [object Object] is set to false. The shape is (expertNum). The number of logical experts to which each physical expert is actually mapped. The data type can be INT32. The must be ND. are supported.
  • [object Object] (aclTensor*, computation input): [object Object] on the device. This parameter is not enabled when [object Object] is set to false. The shape is (expertNum, maxMappingNum), indicating the mapping table of each physical expert or logical expert. The value of [object Object] is less than or equal to 128. The data type must be INT32. The must be ND. are supported.
  • [object Object] (uint32_t, computation input): number of groups. The value must be greater than 0.
  • [object Object] (uint32_t, computation input): number of selected groups. The value must be greater than 0.
  • [object Object] (uint32_t, computation input): number of experts selected from each group for summation. The value must be greater than 0.
  • [object Object] (uint32_t, computation input): number of experts selected in the end. The value must be greater than 0.
  • [object Object] (uint32_t, computation input): activation type. Currently, only 0 (ACTIVATION_SIGMOID) is supported.
  • [object Object] (bool, computation input): whether to normalize the output.
  • [object Object] (float, computation input): coefficient multiplication after normalization.
  • [object Object] (bool, computation input): whether to enable the mapping from physical experts to logical experts.
  • [object Object] (aclTensor*, computation output): [object Object] on the device. The shape is (numToken, topK). The data type is FLOAT32. The must be ND. are supported.
  • [object Object] (aclTensor*, computation output): [object Object] on the device. The shape is (numToken, topK). The data type is INT32. The must be ND. are supported.
  • [object Object] (uint64_t*, output): size of the workspace to be allocated on the device.
  • [object Object] (aclOpExecutor**, output): operator executor, containing the operator computation process.

• Returns:

  [object Object] status code. For details, see .

[object Object]

• Parameters:

  • [object Object] (void *, input): address of the workspace to be allocated on the device.
  • [object Object] (uint64_t, input): size of the workspace to be allocated on the device, which is obtained by calling the first-phase API [object Object].
  • [object Object] (aclOpExecutor *, input): operator executor, containing the operator computation process.
  • [object Object] (aclrtStream, input): stream for executing the task.

• Returns:

  [object Object] status code. For details, see .

[object Object]

• Deterministic computing:

  • [object Object] defaults to a deterministic implementation.

• [object Object] must be an integer multiple of [object Object].

• [object Object] must be less than or equal to [object Object].

• [object Object] must be less than or equal to 128.

• [object Object] must be less than or equal to [object Object].

• [object Object] must be less than or equal to [object Object]/[object Object].

• [object Object] must be less than or equal to 1024.

• [object Object] must be less than or equal to 256.

[object Object]

The following example is for reference only. For details, see .