[object Object][object Object][object Object]undefined[object Object]

• Description: Obtains the input data of [object Object] based on the subscripts stored in [object Object]. If [object Object] data exists, [object Object] is multiplied by [object Object]. Then, this operator computes the cumulative sum and outputs the computation result.

• Formula:

  • If [object Object] is not set to [object Object], the formula is as follows:

    $$T[k] = T[S[k]]$$ $$T[k] = T[k] * P[i][j]$$ $$O[i] = \sum_{k=i*topK}^{(i+1)*topK - 1 } T[k]$$

    where $i \in {0,1,...,tokens-1}$; $j \in {0,1,...,topK-1}$; $k \in {0,1,...,tokens*topK-1}$; [object Object] indicates [object Object]; [object Object] indicates [object Object]; [object Object] indicates [object Object]; [object Object] indicates [object Object]; [object Object] indicates [object Object]; [object Object] indicates [object Object].

  • If [object Object] is set to [object Object], [object Object] is [object Object]. The formula is as follows:

    $$T[i] = T[S[i]]$$ $$O[i] = T[i]$$

    where $i \in {0,1,...,tokens-1}$; [object Object] indicates [object Object]; [object Object] indicates [object Object]; [object Object] indicates [object Object]; [object Object] indicates [object Object].

[object Object]

Each operator has calls. First, [object Object] is called to obtain the workspace size required for computation and the executor that contains the operator computation process. Then, [object Object] is called to perform computation.

• [object Object]

• [object Object]

[object Object]

• Parameters:

  • [object Object] (aclTensor*, computation input): input data. The shape is (tokens_num* topK_num, hidden_size). The data type can be BFLOAT16, FLOAT16, or FLOAT32. The can be ND. Non-contiguous input is supported.

  • [object Object] (aclTensor*, computation input): position of the data to be computed in [object Object]. The shape is (tokens_num* topK_num). The value range is [0, tokens_num * topK_num – 1], and there is no duplicate index. The data type can be INT32. The can be ND. Non-contiguous input is supported.

  • [object Object] (aclTensor*, optional computation input): optional input. When [object Object] is passed, the value of [object Object] equals the second dimension of [object Object]. When [object Object] is not passed, the value of [object Object] is [object Object]. The shape is ( tokens_num, topK_num). The data type can be BFLOAT16, FLOAT16, or FLOAT32. The can be ND. Non-contiguous input is supported.

  • [object Object] (bool, computation input): [object Object] indicates that [object Object] is enabled, and [object Object] indicates that [object Object] is disabled. For details about [object Object], see the [object Object] parameter. Currently, only [object Object] is supported.

  • [object Object] (aclIntArray*, computation input): This parameter takes effect only when [object Object] is set to [object Object]. Otherwise, no operation is performed on this parameter. When [object Object] is set to [object Object], the shape of [object Object] is represented as [object Object]. Currently, only [object Object] is supported.

  • [object Object] (aclTensor*, computation output): output result. When [object Object] is set to [object Object], the shape is ( tokens_num, hidden_size). When [object Object] is set to [object Object], the shape is the same as that of [object Object]. The data type is the same as that of [object Object], supporting BFLOAT16, FLOAT16, and FLOAT32. The can be ND. Non-contiguous output is not supported.

  • [object Object] (uint64_t*, output): size of the workspace to be allocated on the device.

  • [object Object] (aclOpExecutor**, output): operator executor, containing the operator computation process.

• Returns:

  [object Object] status code. For details, see .

  [object Object]

[object Object]

• Parameters:

  • [object Object] (void*, input): address of the workspace to be allocated on the device.
  • [object Object] (uint64_t, input): size of the workspace to be allocated on the device, which is obtained by calling the first-phase API [object Object].
  • [object Object] (aclOpExecutor*, input): operator executor, containing the operator computation process.
  • [object Object] (aclrtStream, input): stream for executing the task.

• Returns:

  [object Object] status code. For details, see .

[object Object]

• Deterministic computation:
  • [object Object] defaults to deterministic implementation.
• [object Object]Atlas A2 training products/Atlas A2 inference products[object Object] and [object Object]Atlas A3 training products/Atlas A3 inference products[object Object]: The value of [object Object] is less than or equal to [object Object].
• [object Object]Atlas inference products[object Object]:
  • The data types supported by [object Object] and [object Object] are FLOAT16 and FLOAT32.
  • The value of [object Object] is less than or equal to [object Object].
  • The value of [object Object] must be a multiple of 128 and less than [object Object].

[object Object]

The following example is for reference only. For details, see .