[object Object][object Object][object Object]undefined[object Object]

• Description: Implements backward propagation of normalization (Norm), Rotary Position Embedding (RoPE), and feature concatenation (Concat) for [object Object], [object Object], and [object Object] in the (multi-modal) transformer attention mechanism.

  • Currently, Norm supports layer normalization (LayerNorm) and layer normalization with affine transformation parameters (AFFINE LayerNorm).
  • RoPE supports the Interleave and Half types.

• Formulas:

  • Backward propagation of LayerNorm:
  $$\frac{\partial L}{\partial x} = \text{rstd} \cdot \Bigg( \frac{\partial L}{\partial y} - \text{Mean}\left( \frac{\partial L}{\partial y} \right) - \hat{x} \cdot \text{Mean}\left( \frac{\partial L}{\partial y} \odot \hat{x} \right) \Bigg) \quad \quad \quad \quad \quad \quad \quad \text{[Mean over headDim dimension]}$$
  • Backward propagation of LayerNorm (with affine transformation parameters):
  $$\left\{ \begin{aligned} \frac{\partial L}{\partial \beta} &= \sum_{B, S, H} \frac{\partial L}{\partial y}, &\quad \text{[Sum over batch, seq, headNum dimensions]} \\ \frac{\partial L}{\partial \gamma} &= \sum_{B, S, H} \frac{\partial L}{\partial y} \odot \hat{x}, &\quad \text{[Element-wise product accumulation]} \\ \frac{\partial L}{\partial x} &= \text{rstd} \cdot \Bigg( \frac{\partial L}{\partial \hat{x}} - \text{Mean}\left( \frac{\partial L}{\partial \hat{x}} \right) - \hat{x} \cdot \text{Mean}\left( \frac{\partial L}{\partial \hat{x}} \odot \hat{x} \right) \Bigg) &\quad \text{[Mean over headDim dimension]} \\ \end{aligned} \right\}$$
  • Where (μ is the mean value, and σ[object Object]2[object Object] is the variance):
  $$\hat{x} = \frac{x - \mu}{\sqrt{\sigma^2 + \epsilon}}, \quad \quad \frac{\partial L}{\partial \hat{x}} = \frac{\partial L}{\partial y} \odot \gamma, \quad \quad \text{rstd} = \frac{1}{\sqrt{\sigma^2 + \epsilon}}$$
  • Backward propagation of RoPE (Interleave):
  $$\frac{\partial L}{\partial x} = \frac{\partial L}{\partial y} \cdot \text{cos} + Interleave({\frac{\partial L}{\partial y} \cdot \text{sin}}) \odot \text{negMask}$$
  • Backward propagation of RoPE (Half):
  $$\frac{\partial L}{\partial x} = \frac{\partial L}{\partial y} \cdot \text{cos} + Half({\frac{\partial L}{\partial y} \cdot \text{sin}}) \odot \text{negMask}$$
  • Interleave() indicates that elements at odd and even positions in the headDim dimension are interleaved. Half() indicates that the second-half and first-half elements in the headDim dimension are interleaved. For example, if x = [0,1,2,3,4,5,6,7], Interleave(x) = [1,0,3,2,5,4,7,6] and Half(x) = [4,0,5,1,6,2,7,3]. The number of elements in negMask is the value of headDim, with even positions being 1 and odd positions being -1, that is, (1, -1, 1, -1, 1, ...).

[object Object]

Each operator has calls. First, [object Object] is called to obtain the input parameters and compute the required workspace size based on the process. Then, [object Object] is called to perform computation.

[object Object]

• Parameters

  [object Object]

• Returns

  [object Object]: status code. For details, see .

  The first-phase API implements input parameter verification. The following errors may be thrown.

  [object Object]

[object Object]

• Parameters

  [object Object]

• Returns

  [object Object]: status code. For details, see .

[object Object]

• Deterministic computation:
  • [object Object] defaults to a non-deterministic implementation. You can call [object Object] to enable deterministic computation.

[object Object]

• Single-aclnn-operator calling

  The following single-aclnn-operator calling example is for reference only. For details, see .

  [object Object]