[object Object][object Object][object Object]undefined[object Object]

• Description: This operator is used for permutation computation of MoE. It broadcasts and sorts tokens based on indexes ([object Object]).
• Formula:

  • When [object Object] is set to [object Object]:

    $$sortedIndicesFirst=argSort(Indices)$$ $$sortedIndicesOut=argSort(sortedIndicesFirst)$$ $$permuteTokens[sortedIndicesFirst[i]]=tokens[i//topK]$$

  • When [object Object] is set to [object Object]:

    $$permuteTokensOut[i]=tokens[Indices[i]]$$ $$sortedIndicesOut=Indices$$

[object Object]

Each operator has calls. First, [object Object] is called to obtain the workspace size required for computation and the executor that contains the operator computation process. Then, [object Object] is called to perform computation.

• [object Object]
• [object Object]

[object Object]

• Parameters:

  • [object Object] (aclTensor*, computation input): input token. It must be a tensor with two or more dimensions. The size of the first dimension is specified by [object Object]. The data type can be FLOAT16, BFLOAT16, or FLOAT32. are supported. The must be ND.
  • [object Object] (aclTensor*, computation input): input indices. The shape must be 2D or 1D. When [object Object] is set to [object Object], it indicates the topK processing expert indexes corresponding to each input token. The shape is (num_tokens, topK) or (num_tokens). When [object Object] is set to [object Object], it indicates the token indexes (not supported currently) selected by each expert. The data type can be INT32 or INT64. are supported. The must be ND. The number of elements must be greater than or equal to [object Object] and less than [object Object]. (The maximum or minimum values of a 32-bit integer or a 64-bit integer is supported.)
    • [object Object]Atlas A2 training products/Atlas A2 inference products[object Object] and [object Object]Atlas A3 training products/Atlas A3 inference products[object Object]: The value of [object Object] is less than or equal to [object Object].
  • [object Object] (int64_t, computation input): number of valid output tokens. If this parameter is set to [object Object], no token is deleted. If this parameter is not set to [object Object], the tokens sorted by [object Object] and exceeding the value specified by [object Object] are dropped. If this parameter is set to a negative number, tokens are processed according to negative slice index rules.
  • [object Object] (bool, computation input): If [object Object] is set to [object Object], [object Object] has been padded with the token indexes selected by each expert. In this case, [object Object] is not sorted. Currently, [object Object] can only be set to [object Object].
  • [object Object] (aclTensor*, computation output): tokens that are extended and ordered based on [object Object]. The value must be a tensor with two or more dimensions. The size of the first dimension is min(num_tokens * topK, numOutTokens). The product of all non-first dimensions is the same as the corresponding product in tokens (excluding the first dimension). The data type is the same as that of [object Object], supporting FLOAT16, BFLOAT16, and FLOAT32. are not supported. The must be ND.
  • [object Object] (aclTensor*, computation output): mapping between [object Object] and [object Object]. The value must be a 1D tensor with shape (num_tokens*topK,). The data type is INT32. are not supported. The must be ND.
  • [object Object] (uint64_t*, output): size of the workspace to be allocated on the device.
  • [object Object] (aclOpExecutor**, output): operator executor, containing the operator computation process.

• Returns:

  [object Object] status code. For details, see .

[object Object]

• Parameters:

  • [object Object] (void*, input): address of the workspace to be allocated on the device.
  • [object Object] (uint64_t, input): size of the workspace to be allocated on the device, which is obtained by calling the first-phase API [object Object].
  • [object Object] (aclOpExecutor*, input): operator executor, containing the operator computation process.
  • [object Object] (aclrtStream, input): stream for executing the task.

• Returns:

  [object Object] status code. For details, see .

[object Object]

• Deterministic computation:

  • [object Object] defaults to deterministic implementation.

• [object Object] requires that the number of elements be less than [object Object], and the value ranges from [object Object] to [object Object] (excluded). (The maximum or minimum value of int-32 or int-64 integer is supported. If the value is not within the range, the sorting result is incorrect.)

• [object Object] cannot be set to [object Object].

• [object Object]Atlas A2 training products/Atlas A2 inference products[object Object] and [object Object]Atlas A3 training products/Atlas A3 inference products[object Object]: The value of [object Object] is less than or equal to [object Object].

[object Object]

The following example is for reference only. For details, see .