[object Object]

[object Object][object Object]undefined[object Object]

• API function: Performs single-channel rotary positional encoding computation. Compared with , this API introduces the input parameter [object Object]. In recommended scenarios, the performance benefits can be obtained by inputting the rotation encoding matrix. Select the appropriate API based on your requirements.

• Recommended usage scenarios for [object Object]:

  • The interleave mode is used, and B x N x S > 28800.
  • The half mode is recommended only in the following scenarios: When the input matrix x needs to be divided into multiple parts in the last dimension, aclnnRotaryPositionEmbedding needs to be called for each part to calculate the rotation position encoding. You can construct a rotation encoding matrix to obtain performance benefits, for example, if the layout of x is BSND and needs to be divided into three parts: If x is divided into three parts, $x = [x1|x2|x3]_{(dim=4)} ∈ R^{B×S×N×D}, x1 ∈ R^{B×S×N×D1},x2 ∈ R^{B×S×N×D2},x3 ∈ R^{B×S×N×D3}, where D = D1 + D2 + D3$, you can construct a rotate matrix to call the aclnnRotaryPositionEmbeddingV2 API once to complete the rotary position encoding calculation of x. The rotate matrix is constructed as follows: $$rotate = diag(rotate1, rotate2, rotate3) = \begin{pmatrix}rotate1&0&0\\0&rotate2&0\\0&0&rotate3\\\end{pmatrix}$$ [object Object], [object Object], and [object Object] represent the rotary encoding matrices corresponding to [object Object], [object Object], and [object Object], respectively. For details about constructing a single rotary encoding matrix, see the examples.

• Formulas:

  • [object Object]Atlas A3 training products/Atlas A3 inference products[object Object] and [object Object]Atlas A2 training products/Atlas A2 inference products[object Object]:

    • Do not pass the rotate parameter (recommended for 1D in half mode).

      (1) Half mode (mode=0):

      $$x1 = x[..., : x.shape[-1] // 2]$$ $$x2 = x[..., x.shape[-1] // 2 :]$$ $$x\_rotate = torch.cat((-x2, x1), dim=-1)$$ $$y = x * cos + x\_rotate * sin$$

      (2) Interleave mode (mode=1):

      $$x1 = x[..., ::2].view(-1, 1)$$ $$x2 = x[..., 1::2].view(-1, 1)$$ $$x\_rotate = torch.cat((-x2, x1), dim=-1).view(x.shape[0], x.shape[1], x.shape[2], x.shape[3])$$ $$y = x * cos + x\_rotate * sin$$

    • When the rotate parameter is passed (the rotate matrix is generated by the developer. For details, see the calling example):

      $$x\_rotate = x @ rotate$$ $$y = x * cos + x\_rotate * sin$$

[object Object]

Each operator has calls. First, [object Object] is called to obtain the input parameters and compute the required workspace size based on the process. Then, [object Object] is called to perform computation.

[object Object]

• Parameters:

  [object Object]

  • Restrictions on the mode parameter:

    • [object Object]Atlas A3 training series/Atlas A3 inference series[object Object], [object Object]Atlas A2 training series/Atlas A2 inference series[object Object];: 0=half, 1=interleave. The restrictions on the mode parameter of the V2 API are the same as those of the V1 API. You can select a proper rotation mode for generating the auxiliary matrix based on the mode in the calling example.
    • Ascend 950PR/Ascend 950DT: 2=quarter, 3=interleave-half.

  • Currently, the rotate parameter supports the BFLOAT16, FLOAT16, and FLOAT32 types.

• Returns

  [object Object]: status code. For details, see .

  The first-phase API implements input parameter validation. The following error codes may be returned.

  [object Object]

[object Object]

• Parameters

  [object Object]

• Returns:

  [object Object]: status code. For details, see .

[object Object]

• [object Object]Atlas A3 training products/Atlas A3 inference products[object Object] and [object Object]Atlas A2 training products/Atlas A2 inference products[object Object]:

  The input tensor x supports BNSD, BSND, and SBND layouts. The D dimension sizes of the input tensor x, cos, sin, and output tensor y must be the same, and the value must be less than 896 and a multiple of 2. The shapes of the input tensor x and output tensor y must be the same. The shapes of the input tensors cos and sin must be the same.

  • half mode:
    • B, N < 1000;
    • When x is BNSD, cos and sin support 11SD, B1SD, and BNSD.
      • When (D/2) % (32/inputDtypeSize) == 0, B N <= S 8 must be met.
      • When (D/2) % (32/inputDtypeSize)!= 0, B N 2 <= (S + coreNum -1) / coreNum or D >= 80 must be met.
    • When x is BSND, cos and sin support 1S1D, BS1D, and BSND.
    • When x is SBND, cos and sin support S11D, SB1D, and SBND.
  • interleave mode:
    • B * N < 1000
    • When x is BNSD, cos and sin support 11SD.
    • When x is BSND, cos and sin support 1S1D.
    • When x is SBND, cos and sin support S11D.

[object Object]

The following example is for reference only. For details, see .

• Example of [object Object] auxiliary matrix generation: