[object Object][object Object][object Object]undefined[object Object]

• Function: Rearranges data of the selectedBlockSize size from the key and value based on topkIndices, and then performs the backward output of attention calculation in the training scenario.

• Calculation formula: Rearranges selectedBlockCount pieces of data of the selectedBlockSize size from the keyIn and value based on the input topkIndice. The formula is as follows:

  $$selectedKey\text{ }=\text{ }Gather \left( key,topkIndices \left[ i \left] \left) ,\text{ }0\text{ } < =i < \text{ }selectBlockCount\right. \right. \right. \right.$$ $$selectedValue\text{ }=\text{ }Gather \left( value,topkIndices \left[ i \left] \left) ,\text{ }0\text{ } < =i < \text{ }selectBlockCount\right. \right. \right. \right.$$

[object Object]

Phase 1: Calculate $dP$ and $dV$ according to the matrix multiplication derivative rules.

[object Object]$$dP\mathop{{}}\nolimits_{{t,:}}=dO\mathop{{}}\nolimits_{{t,:}}\text{@}V\mathop{{}}\nolimits^{{T}}$$ $$dV \left[ u \left] =P\mathop{{}}\nolimits_{{T}}^{{t,:}}\text{@}dO\mathop{{}}\nolimits_{{t,:}}\right. \right.$$ [object Object]

Phase 2: Calculate $dS$.

[object Object]$$d\mathop{{S}}\nolimits_{{t,:}}= \left[ P\mathop{{}}\nolimits_{{t,:}}@ \left( dP\mathop{{}}\nolimits_{{t,:}}-FlashSoftmaxGrad \left( dO,O \left) \left) \right] \right. \right. \right. \right.$$ [object Object]

Phase 3: Calculate $dQ$ and $dK$.

[object Object]$$d\mathop{{Q}}\nolimits_{{t,:}}=d\mathop{{S}}\nolimits_{{t,:}}@K \left[ u \left] \mathop{{}}\nolimits_{{:t,:}}/\sqrt{{d\mathop{{}}\nolimits_{{k,:}}}}\right. \right.$$ $$dK \left[ u \left] \mathop{{}}\nolimits_{{:t,:}}=dS\mathop{{}}\nolimits_{{t,:t}}\mathop{{}}\nolimits^{{T}}\text{@}Q/\sqrt{{d\mathop{{}}\nolimits_{{t,:}}}}\right. \right.$$ [object Object]

Each operator has calls. First, [object Object] is called to obtain the workspace size required for computation and the executor that contains the operator computation process. Then, [object Object] is called to perform computation.

[object Object]

• Parameters

  [object Object]

• Return Value

  [object Object]: status code. For details, see .

  The first-phase API implements input parameter verification. The following errors may be thrown.

  [object Object]

[object Object]

• Parameters

  [object Object]

• Return Value

  [object Object]: status code. For details, see .

[object Object]

• Deterministic computation:

  • By default, aclnnSparseFlashAttentionGrad is implemented in non-deterministic mode. Deterministic computing can be enabled by using aclrtCtxSetSysParamOpt.

• Common Constraints

  • Handling of scenarios where the input parameter is empty:
    • If [object Object] is an empty tensor, the result is returned directly.
  • Currently, only the scenario where the value and key are the same is supported.

• Mask

  [object Object]

• Specification Restrictions

  [object Object]

[object Object]

The following example is for reference only (using [object Object]Atlas A2 training products/Atlas A2 inference products[object Object] as examples). For details, see .