[object Object]

[object Object][object Object]undefined[object Object]

• Description: The RmsNorm operator is a normalization operation commonly used in foundation models. Compared with the LayerNorm operator, the RmsNorm operator removes the part of subtracting the mean value. The DynamicQuant operator is used to perform symmetric dynamic quantization on the input tensor. The AddRmsNormDynamicQuant operator fuses the Add operator before RmsNorm and the normalized output of RmsNorm to one or two DynamicQuant operators to reduce move-in and move-out operations. Compared with aclnnAddRmsNormDynamicQuant, aclnnAddRmsNormDynamicQuantV2 adds the betaOptional parameter (beta in the formula) to the RmsNorm calculation process and adds the outputMaskOptional parameter to the output configuration, which is used to configure whether to output the quantization result at the corresponding position.

• Formula:

  $$x=x_{1}+x_{2}$$ $$y = \operatorname{RmsNorm}(x)=\frac{x}{\operatorname{Rms}(\mathbf{x})}\cdot gamma+beta, \quad \text { where } \operatorname{Rms}(\mathbf{x})=\sqrt{\frac{1}{n} \sum_{i=1}^n x_i^2+epsilon}$$ $$input1 =\begin{cases} y\cdot smoothScale1Optional & \ \ smoothScale1Optional \\ y & !\ smoothScale1Optional \end{cases}$$ $$input2 =\begin{cases} y\cdot smoothScale2Optional & \ \ smoothScale2Optional \\ y & !\ smoothScale2Optional \end{cases}$$ $$scale1Out=\begin{cases} row\_max(abs(input1))/127 & (outputMask[0]=True\ ||\ !outputMask) \& y1Out is INT8. row\_max(abs(input1))/7 & (outputMask[0]=True\ ||\ !outputMask) \& y1Out is INT4. Invalid output & outputMask[0]=False \end{cases}$$ $$y1Out=\begin{cases} round(input1/scale1Out) & outputMask[0]=True\ ||\ !outputMask \\ Invalid output & outputMask[0]=False \end{cases}$$ $$scale2Out=\begin{cases} row\_max(abs(input2))/127 & (outputMask[1]=True\ ||\ (!outputMask\ \&\ smoothScale1Optional\ \&\ smoothScale2Optional)) \& y2Out is INT8. row\_max(abs(input2))/7 & (outputMask[1]=True\ ||\ (!outputMask\ \&\ smoothScale1Optional\ \&\ smoothScale2Optional)) \& y2Out is INT4. Invalid output & outputMask[1]=False\ ||\ (!outputMask\ \&\ smoothScale1Optional\ \&\ !smoothScale2Optional) \end{cases}$$ $$y2Out=\begin{cases} round(input2/scale2Out) & outputMask[1]=True\ ||\ (!outputMask\ \&\ smoothScale1Optional\ \&\ smoothScale2Optional)\\ Invalid output & outputMask[1]=False\ ||\ (!outputMask\ \&\ smoothScale1Optional\ \&\ !smoothScale2Optional) \end{cases}$$

  In the formula, row_max indicates that the maximum value of each row is calculated.

[object Object]

Each operator has calls. First, [object Object] is called to obtain the input parameters and compute the required workspace size based on the process. Then, `aclnnAddRmsNormDynamicQuantV2 is called to perform computation.

[object Object]

• Parameters

  [object Object]

• Returns:

  [object Object]: status code. For details, see .

  The first-phase API implements input parameter verification. The following errors may be thrown.

  [object Object]

[object Object]

• Parameters

  [object Object]

• Returns:

  aclnnStatus: status code. (For details, see [aclnn Return Codes](../common/aclnn Return Codes md).)

[object Object]

• Data format description: The ND format is recommended for all input and output tensors. If other data formats are used, the framework converts them into the ND format by default for processing.

• When outputMaskOptional is not empty and smoothScale1Optional has a value, outputMaskOptional[0] must be True. When smoothScale2Optional has a value, outputMaskOptional[1] must be True.

• When outputMaskOptional is not empty, outputMaskOptional[0] and outputMaskOptional[1] cannot both be False.

• When outputMaskOptional is empty and smoothScale2Optional has a value, smoothScale1Optional must also have a value.

• The following table describes the data types supported by different product models.

  [object Object]undefined

• Deterministic compute:

  • aclnnAddRmsNormDynamicQuantV2 defaults to a deterministic implementation.

[object Object]

The following example is for reference only. For details, see .