[object Object][object Object][object Object]undefined[object Object]

• Description: Implements selected attention computation in the Native Sparse Attention (NSA) algorithm for training scenarios.

• Formulas: The forward propagation formulas for selected attention are as follows:

  $$selected\_key = Gather(key, topk\_indices[i]),0<=i<selected\_block\_count \\ selected\_value = Gather(value, topk\_indices[i]),0<=i<selected\_block\_count$$ $$attention\_out = Softmax(Mask(scale * (query @ selected\_key^T), atten\_mask)) @ selected\_value$$

[object Object]

Each operator has calls. First, [object Object] is called to obtain the workspace size required for computation and the executor that contains the operator computation process. Then, [object Object] is called to perform computation.

[object Object]

• Parameters:

  [object Object]

• Returns:

  [object Object]: status code. For details, see .

  The first-phase API implements input parameter verification. The following errors may be thrown.

  [object Object]

[object Object]

• Parameters:

  [object Object]

• Returns:

  [object Object]: status code. For details, see .

[object Object]

• Deterministic computation:

  • [object Object] defaults to a deterministic implementation.

• When this API is used together with PyTorch, ensure that the CANN package versions match the PyTorch package versions.

• [object Object] of the input [object Object], [object Object], and [object Object] must be the same. That is, the values of the input [object Object] and [object Object] must be the same.

• [object Object] ([object Object]) of the input [object Object], [object Object], and [object Object] must satisfy [object Object].

• The data types of the input [object Object], [object Object], and [object Object] must be the same.

• [object Object] of the input [object Object], [object Object], and [object Object] must be the same.

• The value range of [object Object] is [1, 128]. The total size of the selected blocks ([object Object]) must be less than [object Object] (8K).

• When the layout is [object Object], [object Object] of each batch must be greater than [object Object].

• The [object Object] values of the input [object Object] and [object Object]/[object Object] can be different, but [object Object] must be a non-zero integer, which is called [object Object] ([object Object]), and [object Object] must be less than or equal to [object Object].

• If [object Object] is [object Object], the [object Object] parameter does not take effect and all tokens are computed.

• The following uses the [object Object] [object Object] as an example to describe the restrictions on the data shape. (Note: T is the sum of [object Object] in all batches. When [object Object] in each batch is the same, [object Object] = [object Object].)

  • [object Object] ([object Object]): The value ranges from 1 to 1024.
  • [object Object] ([object Object]): The value ranges from 1 to 128.
  • [object Object] ([object Object]): The value ranges from 1 to 32.
  • [object Object] ([object Object]): The value ranges from 1 to 128K. In addition, [object Object] must be greater than or equal to the product of [object Object] and [object Object], and be an integer multiple of [object Object].
  • [object Object] ([object Object]): [object Object] is [object Object] and [object Object] is [object Object].

[object Object]

The following single-aclnn-operator calling example is for reference only. For details, see .