[object Object][object Object][object Object]undefined[object Object]

• Description: Completes the Attention calculation for the Swin-Transformer scenario, supporting INT8 quantization compared with the [object Object] operator.
• Formula:

$$out= Softmax(QK^T + bias1 + bias2)V$$ [object Object]

Each operator has calls. First call [object Object] to obtain the required workspace size for computation and the executor that includes the operator's computation process. Then, call [object Object] to perform the computation.

• [object Object]
• [object Object]

[object Object]

• Parameters:

  • [object Object] (aclTensor*, computation input): Represents the query tensor of the input sample, the aclTensor on the Device side, Q in the formula, supports four dimensions, the input dimension [N,C,S,H] needs to be consistent with key and value, where N represents the batch size, C is the channel depth, S is the sequence length, H is the headNum, S<=1024, H=32/64, and NC dimensions support any value. The data type supports INT8, while are not allowed. The supports ND.
  • [object Object] (aclTensor*, computation input): Represents the feature tensor for each position of the input sample, an aclTensor on the device side, denoted as K in the formula. It supports four-dimensional input with dimensions [N,C,S,H], which must be consistent with query and value. S<=1024, H=32/64, and NC dimensions support arbitrary values. The data type supports INT8, while are not allowed. The supports ND.
  • [object Object] (aclTensor*, computation input): Represents the tensor of values at each position after computing attention, an aclTensor on the Device side, corresponding to V in the formula. It supports four dimensions, with the input dimensions [N,C,S,H] needing to be consistent with query and key, where S<=1024, H=32/64, and NC dimensions can be any value. The data type supports INT8, while are not allowed. The supports ND.
  • [object Object] (aclTensor*, computation input): Represents the scale tensor for quantization after attention softmax normalization and an aclTensor on the device side, supporting 2D dimensions with input dimensions [1,S], S<=1024. The data type supports FLOAT16, while are not allowed. The supports ND.
  • [object Object] (aclTensor*, computation input): Represents the scale tensor for dequantization during attention computation, an aclTensor on the Device side. It supports two-dimensional tensors with input dimensions [1,S], where S<=1024. The data type supports UINT64, while are not allowed. The supports ND.
  • [object Object] (aclTensor*, computation input): Represents the scale tensor for dequantization of the output after attention computation, an aclTensor on the Device side, supporting two-dimensional dimensions, input dimension [1,H], H=32/64. The data type supports UINT64, while are not allowed. The supports ND.
  • [object Object] (aclTensor*, computation input): Represents the offset tensor for quantization after attention softmax normalization, a Device-side aclTensor, supporting two-dimensional dimensions, input dimensions [1,S], S<=1024. The data type supports FLOAT16, while are not allowed. The supports ND.
  • [object Object] (aclTensor*, computation input): Represents the offset tensor for dequantization during attention computation, an aclTensor on the Device side. It supports two-dimensional tensors with input dimensions [1,S], where S<=1024. The data type supports INT32, while are not allowed. The supports ND.
  • [object Object] (aclTensor*, computation input): Represents the offset tensor for dequantization of the output after attention computation on the Device side. The aclTensor supports two-dimensional dimensions, with input dimensions [1,H], where H=32/64. The data type supports INT32, while are not allowed. The supports ND.
  • [object Object] (aclTensor*, computation input): Device-side aclTensor, bias1 in the formula, supports input of nullptr, or a four-dimensional tensor with dimensions [1,C,S,S], where S<=1024. The C dimension supports any value. The data type supports FLOAT16, while are not allowed. The supports ND.
  • [object Object] (aclTensor*, computation input): Reserved parameter, bias2 in the formula, currently only supports input nullptr.
  • [object Object] (bool, computation input): A bool on the Host side, indicating whether the query is transposed. Currently, only non-transposed (false) is supported.
  • [object Object] (bool, computation input): A bool on the host side, indicating whether the key is transposed. Currently, only non-transposed (false) is supported.
  • [object Object] (bool, computation input): Host-side bool, indicating whether the value is transposed. Currently, only non-transposed (false) is supported.
  • [object Object] (int, computation input): An int on the host side, used to specify the dimension for softmax calculation. Currently, only -1 is supported (that is, the last dimension of the tensor).
  • [object Object] (aclTensor*, computation output): Device-side aclTensor, supports four dimensions, output dimensions [N,C,S,H], S<=1024, H=32/64. The data type supports FLOAT16, while are not allowed. The supports ND.
  • [object Object] (uint64_t*, output): Returns the size of the workspace that needs to be allocated on the device side.
  • [object Object] (aclOpExecutor**, output): Returns the op executor, which includes the operator computation process.

• Returns:

  [object Object]: status code. For details, see .

[object Object]

• Parameters

  • [object Object] (void *, input): The memory address of the workspace allocated on the device side.
  • [object Object] (uint64_t, input): The size of the workspace allocated on the device side, obtained by the first interface [object Object].
  • [object Object] (aclOpExecutor *, input): The op executor, which includes the operator computation process.
  • [object Object] (aclrtStream, input): Specifies the stream for task execution.

• Returns

  [object Object] status code. For details, see .

[object Object]

• When the QKV input dimension is [N,C,S,H], S<=1024, H=32/64, the NC dimension supports any value.
• Transposed QKV input with dimensions [N,C,S,H] is not supported.
• Only supports asymmetric quantization.
• The feature of adding bias2 is not supported.
• Only supports the softmax operation on the last dimension of QK^T + bias1 + bias2.

[object Object]

The following example is for reference only. For details, see .