[object Object][object Object][object Object]undefined[object Object]

• Description: Uses the FloydAttention algorithm to perform multidimensional self-attention computation in training scenarios.

• Formulas:

  The forward propagation formula for attention is as follows:

  $$weights = Softmax(attenMask + scale*(einsum(query, key1^T) + einsum(query, key2^T)))$$ $$attention\_out = einsum(weights, value1) + einsum(weights, value2)$$

[object Object]

Each operator has calls. First, [object Object] is called to obtain the workspace size required for computation and the executor that contains the operator computation process. Then, [object Object] is called to perform computation.

[object Object]

• Parameters

  [object Object]

• Returns:

  [object Object]: status code. For details, see .

  The first-phase API implements input parameter validation. The following error codes may be returned.

  [object Object]

[object Object]

• Parameters

  [object Object]

• Returns:

  [object Object]: status code. For details, see .

[object Object]

• When this API is used together with PyTorch, ensure that the CANN package versions match the PyTorch package versions.

• Shape constraints:

  • B: The value ranges from 1 to 2K.
  • [object Object]: The value ranges from 1 to 256.
  • [object Object]: The value ranges from 16 to 1M and must be a multiple of 16.
  • [object Object]: The value ranges from 128 to 1M and must be a multiple of 128.
  • [object Object]: The value ranges from 128 to 1M and must be a multiple of 128.
  • [object Object]: The value can be 32, 64, or 128.

• The axis 0, 2, or 4 of query must be the same as that of key1.

• The shapes of key1 and value1 must be the same.

• The shapes of key2 and value2 must be the same.

• The shapes of softmaxMax and softmaxSum must be the same.

• [object Object]: Only [object Object], [object Object], and [object Object] are supported.

• Due to the restriction of underlying instructions, when M x D >= 65536 or K x D >= 65536, the performance deteriorates significantly. In this case, you are advised to use small operators to replace the implementation.

[object Object]

The following example is for reference only. For details, see .