[object Object][object Object][object Object]undefined[object Object]

• Description: Implements backward propagation of normalization (Norm), Rotary Position Embedding (RoPE), and feature concatenation (Concat) for [object Object], [object Object], and [object Object] in the (multi-modal) transformer attention mechanism.

  • Currently, Norm supports layer normalization (LayerNorm) and layer normalization with affine transformation parameters (AFFINE LayerNorm).
  • RoPE supports the Interleave and Half types.

• Formulas:

  • Backward propagation of LayerNorm:

    $$\frac{\partial L}{\partial x} = \text{rstd} \cdot \Bigg( \frac{\partial L}{\partial y} - \text{Mean}\left( \frac{\partial L}{\partial y} \right) - \hat{x} \cdot \text{Mean}\left( \frac{\partial L}{\partial y} \odot \hat{x} \right) \Bigg) \quad \quad \quad \quad \quad \quad \quad \text{[Mean over headDim dimension]}$$

  • Backward propagation of LayerNorm (with affine transformation parameters):

    $$\left\{ \begin{aligned} \frac{\partial L}{\partial \beta} &= \sum_{B, S, H} \frac{\partial L}{\partial y}, &\quad \text{[Sum over batch, seq, headNum dimensions]} \\ \frac{\partial L}{\partial \gamma} &= \sum_{B, S, H} \frac{\partial L}{\partial y} \odot \hat{x}, &\quad \text{[Element-wise product accumulation]} \\ \frac{\partial L}{\partial x} &= \text{rstd} \cdot \Bigg( \frac{\partial L}{\partial \hat{x}} - \text{Mean}\left( \frac{\partial L}{\partial \hat{x}} \right) - \hat{x} \cdot \text{Mean}\left( \frac{\partial L}{\partial \hat{x}} \odot \hat{x} \right) \Bigg) &\quad \text{[Mean over headDim dimension]} \\ \end{aligned} \right\}$$

  • Where (μ is the mean value, and σ[object Object]2[object Object] is the variance):

    $$\hat{x} = \frac{x - \mu}{\sqrt{\sigma^2 + \epsilon}}, \quad \quad \frac{\partial L}{\partial \hat{x}} = \frac{\partial L}{\partial y} \odot \gamma, \quad \quad \text{rstd} = \frac{1}{\sqrt{\sigma^2 + \epsilon}}$$

  • Backward propagation of RoPE (Interleave):

    $$\frac{\partial L}{\partial x} = \frac{\partial L}{\partial y} \cdot \text{cos} + Interleave({\frac{\partial L}{\partial y} \cdot \text{sin}}) \odot \text{negMask}$$

  • Backward propagation of RoPE (Half):

    $$\frac{\partial L}{\partial x} = \frac{\partial L}{\partial y} \cdot \text{cos} + Half({\frac{\partial L}{\partial y} \cdot \text{sin}}) \odot \text{negMask}$$

  • Interleave() indicates that the odd and even positions in the headDim dimension are alternately rearranged, and Half() indicates that the second half and the first half of the elements in the headDim dimension are alternately rearranged. For example, if x = [0,1,2,3,4,5,6,7], then Interleave(x) = [1,0,3,2,5,4,7,6] and Half(x) = [4,0,5,1,6,2,7,3]. negMask is the length of headDim. The even bit is 1 and the odd bit is -1, that is, (1, -1, 1, -1, 1,...).

[object Object]

Each operator has calls. First, [object Object] is called to obtain the input parameters and compute the required workspace size based on the process. Then, [object Object] is called to perform computation.

[object Object]

• Parameters

  [object Object]

• Returns

  [object Object]: status code. For details, see .

  The first-phase API implements input parameter verification. The following errors may be thrown.

  [object Object]

[object Object]

• Parameters

  [object Object]

• Returns

  [object Object]: status code. For details, see .

[object Object]

• Deterministic computation:
  • [object Object] defaults to a non-deterministic implementation. You can call [object Object] to enable deterministic computation.

[object Object]

• Single-aclnn-operator calling

  The following is an example of aclnn single-operator calling. For details about the compilation and execution process, see .

  [object Object]