[object Object][object Object][object Object]undefined[object Object]

• Description: Implements compression in the KV sequence dimension in the training scenario by leveraging NSA Compress algorithm to reduce long-context attention computation.

• Formula:

  The forward propagation formula of NSA Compress is as follows:

$$\tilde{K}_t^{\text{cmp}} = f_K^{\text{cmp}}(k_{:t}) = \left\{ \varphi(k_{id+1:id+l}) \bigg| 0 \leq i \leq \left\lfloor \frac{t-l}{d} \right\rfloor \right\}$$ [object Object]

Each operator has calls. First, [object Object] is called to obtain the workspace size required for computation and the executor that contains the operator computation flow. Then, [object Object] is called to perform computation.

[object Object]

• Parameters

  [object Object]
  • The data layout of [object Object] can be interpreted from multiple dimensions. To be specific, [object Object] ([object Object]) indicates the size of an input sample batch, [object Object] ([object Object]) indicates the length of the input sample sequence, [object Object] indicates the total length of all input sample sequences ([object Object] of all batches), [object Object] ([object Object]) indicates the size of the hidden layer, [object Object] ([object Object]) indicates the number of heads, and [object Object] ([object Object]) indicates the minimum unit size of the hidden layer ([object Object] = [object Object]/[object Object]).

• Returns:

  [object Object]: status code. For details, see .

  The first-phase API implements input parameter validation. The following error codes may be returned.

  [object Object]

[object Object]

• Parameters

  [object Object]

• Returns:

  [object Object]: status code. For details, see .

[object Object]

• Deterministic computation:
  • [object Object] defaults to a deterministic implementation.
• When this API is used together with PyTorch, ensure that the CANN package versions match the PyTorch package versions.
• [object Object] and [object Object] must meet the broadcast relationship, that is [object Object] = [object Object]. [object Object] and [object Object] cannot be empty.
• Currently, [object Object] can only be set to [object Object], that is, the cumulative sum mode is used for [object Object].
• Currently, actSeqLenOptional cannot be empty.
• Currently, [object Object] supports only [object Object]. In this case, [object Object] must be equal to [object Object].
• [object Object] must be equal to [object Object], and their value must be less than or equal to 128.
• [object Object] must be a multiple of 16, and cannot exceed 256.
• [object Object] must be equal to [object Object]. Their value must be a multiple of 16, and cannot exceed 128.
• [object Object] must be an integer multiple of 16, and [object Object] must be greater than or equal to [object Object].

[object Object]

The following example is for reference only. For details, see .