[object Object][object Object][object Object]undefined[object Object]

• Description: Rearranges [object Object] and [object Object] data blocks with a size of [object Object] based on [object Object], and then computes the backward attention output in the training scenario.

• Formulas:

  Based on the input [object Object], select [object Object] blocks with a size of [object Object] from [object Object] and [object Object] for rearrangement. The formula is as follows:

  $$selectedKey = Gather(key, topkIndices[i]),0<=i<selectedBlockCount \\ selectedValue = Gather(value, topkIndices[i]),0<=i<selectedBlockCount$$

  Then, backward propagation of the attention mechanism is performed. The formula is as follows:

  $$dV=P^TdY$$ $$dQ=\frac{((dS)*K)}{\sqrt{d}}$$ $$dK=\frac{((dS)^T*Q)}{\sqrt{d}}$$

[object Object]

Each operator has calls. First, [object Object] is called to obtain the workspace size required for computation and the executor that contains the operator computation process. Then, [object Object] is called to perform computation.

[object Object]

• Parameters

  [object Object]

• Returns:

  [object Object]: status code. For details, see .

  The first-phase API implements input parameter validation. The following error codes may be returned.

  [object Object]

[object Object]

• Parameters

  [object Object]

• Returns:

  [object Object]: status code. For details, see .

[object Object]

• Deterministic computation:
  • [object Object] defaults to a non-deterministic implementation. You can call [object Object] to enable deterministic computation.
• When this API is used together with PyTorch, ensure that the CANN package versions match the PyTorch package versions.
• [object Object] ([object Object]) of the input [object Object], [object Object], [object Object], [object Object], and [object Object] must be the same.
• [object Object] ([object Object]) of the input [object Object] and [object Object] must be the same.
• [object Object] ([object Object]) of the input [object Object], [object Object], and [object Object] must be the same.
• [object Object] ([object Object]) of the input [object Object], [object Object], and [object Object] must be the same.
• [object Object] of the input [object Object], [object Object], [object Object], [object Object], and [object Object] must be the same.
• The following uses TND of inputLayout as an example to describe the restrictions on the data shape. The values are as follows:
  • [object Object]: The value ranges from 1 to 2M. [object Object] indicates the sum of [object Object] of all batches in [object Object].
  • [object Object]: The value ranges from 1 to 2M. [object Object] indicates the sum of [object Object] of all batches in [object Object]/[object Object].
  • [object Object]: The value ranges from 1 to 2M.
  • [object Object]: The value ranges from 1 to 128. [object Object] indicates [object Object] of [object Object]. [object Object] must be an integer multiple of [object Object].
  • [object Object]: The value ranges from 1 to 128. [object Object] indicates [object Object] of [object Object] and [object Object].
  • [object Object]: The value ranges from 1 to 32. [object Object] = [object Object]/[object Object]
  • [object Object]: The value ranges from 1 to 128K. The value of [object Object] for [object Object] and [object Object] must be greater than or equal to the product of [object Object] and [object Object], and must be an integer multiple of [object Object].
  • [object Object]: The value can be [object Object] or [object Object]. [object Object] ([object Object]) of [object Object] and [object Object] can be different.
  • The value of [object Object] must be less than or equal to 128 and be an integer multiple of 16.
  • The value range of [object Object] is [1, 128]. The total size of the selected blocks ([object Object]) must be less than [object Object] (8K).
  • When the layout is [object Object], [object Object] of each batch must be greater than [object Object].
• The shape of the [object Object] and [object Object] parameters is restricted to [object Object].
• The shape of the [object Object] parameter is restricted to [object Object].

[object Object]

The following example is for reference only. For details, see .