[object Object]

[object Object][object Object]undefined[object Object]

• Description: The RmsNorm operator is a normalization operation commonly used in foundation models. Compared with the LayerNorm operator, the RmsNorm operator removes the part of subtracting the mean value. The DynamicQuant operator is used to perform symmetric dynamic quantization on the input tensor. The AddRmsNormDynamicQuant operator fuses the Add operator before RmsNorm and the normalized output of RmsNorm to one or two DynamicQuant operators to reduce move-in and move-out operations.
• Formula:$$x=x_{1}+x_{2}$$ $$y = \operatorname{RmsNorm}(x)=\frac{x}{\operatorname{Rms}(\mathbf{x})}\cdot gamma, \quad \text { where } \operatorname{Rms}(\mathbf{x})=\sqrt{\frac{1}{n} \sum_{i=1}^n x_i^2+epsilon}$$
  • If smoothScale1Optional and smoothScale2Optional are not input, the outputs y2Out and scale2Out are meaningless. The calculation process is as follows:
  $$scale1Out=row\_max(abs(y))/127$$ $$y1Out=round(y/scale1Out)$$
  • If only smoothScale1Optional is input, the outputs y2Out and scale2Out are meaningless. The calculation process is as follows:
  $$input = y\cdot smoothScale1Optional$$ $$scale1Out=row\_max(abs(input))/127$$ $$y1Out=round(input/scale1Out)$$
  • If both smoothScale1Optional and smoothScale2Optional are input, all the five outputs of the operator are valid. The calculation process is as follows:
  $$input1 = y\cdot smoothScale1Optional$$ $$input2 = y\cdot smoothScale2Optional$$ $$scale1Out=row\_max(abs(input1))/127$$ $$scale2Out=row\_max(abs(input2))/127$$ $$y1Out=round(input1/scale1Out)$$ $$y2Out=round(input2/scale2Out)$$

  row_max indicates that the maximum value of each row is calculated.

[object Object]

Each operator has calls. First, [object Object] is called to obtain the input parameters and compute the required workspace size based on the process. Then, [object Object] is called to perform computation.

[object Object]

• Parameters

  [object Object]
  • [object Object]Atlas A3 training products/Atlas A3 inference products[object Object] and [object Object]Atlas A2 training products/Atlas A2 inference products[object Object]:
    • The output parameters [object Object] and [object Object] support only INT8.

• Returns:

  [object Object]: status code. For details, see .

  The first-phase API implements input parameter verification. The following errors may be thrown.

  [object Object]

[object Object]

• Parameters

  [object Object]

• Returns:

  aclnnStatus: status code. (For details, see [aclnn Return Codes](../common/aclnn Return Codes.md).)

[object Object]

• Description of supported types

  Empty tensors: Empty input and output are supported.

• Data format description

  The ND format is recommended for all input and output tensors. If other data formats are used, the framework converts them into the ND format by default for processing.

• Description of data types supported by different product models

  • [object Object]Atlas A3 training products/Atlas A3 inference products[object Object] and [object Object]Atlas A2 training products/Atlas A2 inference products[object Object]:

    [object Object]undefined

  • Ascend 950PR/Ascend 950DT:

    [object Object]undefined

• Deterministic compute:

  • aclnnAddRmsNormDynamicQuant defaults to a deterministic implementation.

[object Object]

The following example is for reference only. For details, see .