[object Object][object Object][object Object]undefined[object Object]

• API function: updates the outputs of two FlashAttentions based on the max and sum of different softmax operations. The difference between this API and the API is as follows: When the input layout is TND, the softmax-related input data layout in the original RingAttentionUpdate API is BNS8. The RingAttentionUpdateV2 API supports the inputSoftmaxLayout parameter to control whether the softmax-related input data layout is consistent with that of attention (that is, the TND layout is used).
• Formulas:

$$softmax\_max = max(prev\_softmax\_max, cur\_softmax\_max)$$ $$softmax\_sum = prev\_softmax\_sum * exp(prev\_softmax\_max - softmax\_max) + cur\_softmax\_sum * exp(cur\_softmax\_max - softmax\_max)$$ $$attn\_out = prev\_attn\_out * exp(prev\_softmax\_max - softmax\_max) * prev\_softmax\_sum / softmax\_sum + cur\_attn\_out * exp(cur\_softmax\_max - softmax\_max) * cur\_softmax\_sum / softmax\_sum$$ [object Object]

Each operator has calls. First, [object Object] is called to obtain the workspace size required for computation and the executor that contains the operator computation process. Then, [object Object] is called to perform computation.

[object Object]

• Parameters

  [object Object]
  • [object Object]Atlas A2 training products/Atlas A2 inference products[object Object] and [object Object]Atlas A3 training products/Atlas A3 inference products[object Object]: When the input data layout (inputLayoutOptional) is TND, D must be a multiple of 64.

• Returns:

  [object Object]: status code. For details, see .

  The first-phase API implements input parameter validation. The following error codes may be returned.

  [object Object]

[object Object]

• Parameters

  [object Object]

• Returns:

  [object Object]: status code. For details, see .

[object Object]

• Deterministic computation:
  • The deterministic implementation of aclnnRingAttentionUpdateV2 is used by default.
• When [object Object] is TND, [object Object] is required.
• When inputLayoutOptional is set to TND:
  • N:
    • [object Object]Atlas A2 training products/Atlas A2 inference products[object Object] and [object Object]Atlas A3 training products/Atlas A3 inference products[object Object]: N <= 256.
  • D:
    • [object Object]Atlas A2 training products/Atlas A2 inference products[object Object] and [object Object]Atlas A3 training products/Atlas A3 inference products[object Object]: D <= 768 and D is a multiple of 64.
• When inputLayoutOptional is set to TND, inputSoftmaxLayoutOptional takes effect. inputSoftmaxLayoutOptional supports only three types of input: empty string, SBH, and TND.

[object Object]

The following example is for reference only. For details, see .