[object Object]

[object Object][object Object]undefined[object Object]

• Description: Fetches discontinuous tokens from [object Object]/[object Object] and assembles them into a continuous [object Object]/[object Object] sequence based on the [object Object] value in [object Object] and [object Object] of [object Object]/[object Object] from [object Object].

• Computation logic:

  • The first dimension of [object Object]/[object Object] depends on the value of [object Object].
  • If [object Object] is [object Object], the last value in [object Object] is the size of the first dimension of [object Object]/[object Object]: [object Object] = [object Object].
  • If [object Object] is [object Object], the sum of all values in [object Object] is the size of the first dimension of [object Object]/[object Object]: [object Object] = [object Object].

  Restrictions on [object Object] and [object Object]:

  • The size of each token must be less than or equal to 148 KB. For example, for the fp16/bf16 type, num_headshead_size(keyRef/valueRef) is 128576.

• Example:

  [object Object]

[object Object]

Each operator has calls. First, [object Object] is called to obtain the workspace size required for computation and the executor that contains the operator computation process. Then, [object Object] is called to perform computation.

[object Object]

• Parameters

  [object Object]

  • [object Object]Atlas A2 training products/Atlas A2 inference products[object Object] and [object Object]Atlas A3 training products/Atlas A3 inference products[object Object]:

    • The input keyCache, valueCache, keyRef, and valueRef do not support the FLOAT, UINT8, INT16, UINT16, INT32, UINT32, HIFLOAT8, FLOAT8_E5M2 and FLOAT8_E4M3FN data types.
    • The input blockTables, seqLens, and seqOffsetOptional do not support the INT64 data type.

• Returns:

  [object Object]: status code. For details, see .

  The first-phase API implements input parameter verification. The following errors may be thrown:

  [object Object]

[object Object]

• Parameters

  [object Object]

• Returns:

  [object Object]: status code. For details, see .

[object Object]

• Deterministic computation:
  • [object Object] defaults to a deterministic implementation.

[object Object]

The following example is for reference only. For details, see .