[object Object][object Object][object Object]undefined[object Object]

• API function: Updates the outputs of two FlashAttention operations based on their respective maximum and sum softmax values.

• Formula:

$$softmax\_max = max(prev\_softmax\_max, cur\_softmax\_max)$$ $$softmax\_sum = prev\_softmax\_sum * exp(prev\_softmax\_max - softmax\_max) + cur\_softmax\_sum * exp(cur\_softmax\_max - softmax\_max)$$ $$attn\_out = prev\_attn\_out * exp(prev\_softmax\_max - softmax\_max) * prev\_softmax\_sum / softmax\_sum + cur\_attn\_out * exp(cur\_softmax\_max - softmax\_max) * cur\_softmax\_sum / softmax\_sum$$ [object Object]

Each operator has calls. First, [object Object] is called to obtain the workspace size required for computation and the executor that contains the operator computation process. Then, [object Object] is called to perform computation.

[object Object]

• Parameters:

  [object Object]
  • For [object Object]Atlas A2 training products/Atlas A2 inference products[object Object] and [object Object]Atlas A3 training products/Atlas A3 inference products[object Object], when the input data layout (inputLayoutOptional) is TND, D must be a multiple of 64.

• Returns:

  [object Object]: status code. For details, see .

  The first-phase API implements input parameter verification. The following errors may be thrown:

  [object Object]

[object Object]

• Parameters:

  [object Object]

• Returns:

  [object Object]: status code. For details, see .

[object Object]

• Deterministic computing:
  • [object Object] defaults to a deterministic implementation.
  • When [object Object] is TND, [object Object] is required.
  • When inputLayoutOptional is set to TND:
    • N:
      • [object Object]Atlas A2 training products/Atlas A2 inference products[object Object] and [object Object]Atlas A3 training products/Atlas A3 inference products[object Object]: N <= 256.
    • D:
      • [object Object]Atlas A2 training products/Atlas A2 inference products[object Object] and [object Object]Atlas A3 training products/Atlas A3 inference products[object Object]: D <= 768 and D is a multiple of 64.

[object Object]

The following example is for reference only. For details, see .