[object Object][object Object][object Object]undefined[object Object]

• Description: Performs KV compression in the Native Sparse Attention (NSA) inference phase. A new token is generated for each batch during each inference. When the number of tokens in a batch reaches the size of a compression block, the operator compresses the last compression-block-sized tokens in the batch into a compressed token.
• Formulas:

$$compressIdx=(s-compressBlockSize)/stride\\ outputCacheRef[slotMapping[i]] = input[compressIdx*stride : compressIdx*stride+compressBlockSize]*weight[:]$$ [object Object]

Each operator has calls. First, [object Object] is called to obtain the workspace size required for computation and the executor that contains the operator computation process. Then, [object Object] is called to perform computation.

[object Object]

• Parameters

  [object Object]

• Returns

  [object Object]: status code. For details, see .

  The first-phase API implements input parameter verification. The following errors may be thrown.

  [object Object]

[object Object]

• Parameters

  [object Object]

• Returns

  [object Object]: status code. For details, see .

[object Object]

• Deterministic computation:
  • [object Object] defaults to a deterministic implementation.
• The N and D of outputCache are the same as those of input, and must meet the result_len>(blockNum*pageBlockSize-compressBlockSize)/compressStride condition.
• In the page attention scenario, the shape of [object Object] supports [blockNum, pageBlockSize, N, D]. In other scenarios, the shape of [object Object] supports [T, N, D].

[object Object]

The following example is for reference only. For details, see .