[object Object][object Object][object Object]undefined[object Object]

• Description: Implements selected attention computation in the Native Sparse Attention (NSA) algorithm for training scenarios.

• Formulas: The forward propagation formula for selected attention is as follows:

  $$selected\_key = Gather(key, topk\_indices[i]),0<=i<selected\_block\_count \\ selected\_value = Gather(value, topk\_indices[i]),0<=i<selected\_block\_count$$ $$attention\_out = Softmax(Mask(scale * (query @ selected\_key^T), atten\_mask)) @ selected\_value$$

[object Object]

Each operator has calls. First, [object Object] is called to obtain the workspace size required for computation and the executor that contains the operator computation process. Then, [object Object] is called to perform computation.

[object Object]

• Parameters

  [object Object]

• Returns:

  [object Object]: status code. For details, see .

  The first-phase API implements input parameter validation. The following error codes may be returned.

  [object Object]

[object Object]

• Parameters

  [object Object]

• Returns:

  [object Object]: status code. For details, see .

[object Object]

• Deterministic computation:

  • [object Object] defaults to a deterministic implementation.

• When this API is used together with PyTorch, ensure that the CANN package versions match the PyTorch package versions.

• [object Object] of the input [object Object], [object Object], and [object Object] must be the same. That is, the values of the input [object Object] and [object Object] must be the same.

• [object Object] ([object Object]) of the input [object Object], [object Object], and [object Object] must satisfy [object Object].

• The input data types of [object Object], [object Object], and [object Object] must be the same.

• [object Object] of the input [object Object], [object Object], and [object Object] must be the same.

• Currently, sparseMode can be set to 0 or 2.

• The value of [object Object] must be less than or equal to 128 and be an integer multiple of 16.

• The value range of [object Object] is [1, 128]. The total size of the selected blocks ([object Object]) must be less than [object Object] (8K).

• When the layout is [object Object], [object Object] of each batch must be greater than [object Object].

• Currently, inputLayout supports only TND.

• The [object Object] values of the input [object Object] and [object Object]/[object Object] can be different, but [object Object] must be a non-zero integer, which is called [object Object] (group), and [object Object] must be less than or equal to [object Object].

• If [object Object] is [object Object], the [object Object] parameter does not take effect and all tokens are computed.

• The following uses the inputLayout TND as an example to describe the restrictions on the data shape. (Note: T is the sum of the lengths of S in each batch. When S in each batch is the same, T = B x S.) The values are as follows:

  • [object Object] ([object Object]): The value ranges from 1 to 1024.
  • [object Object] ([object Object]): The value ranges from 1 to 128.
  • [object Object] ([object Object]): The value ranges from 1 to 32.
  • [object Object] ([object Object]): The value ranges from 1 to 128K. In addition, [object Object] must be greater than or equal to the product of [object Object] and [object Object], and be an integer multiple of [object Object].
  • [object Object] ([object Object]): [object Object] is [object Object] and [object Object] is [object Object].

[object Object]

The following is an example of aclnn single-operator calling. For details about the compilation and execution process, see .