[object Object]

[object Object][object Object]undefined[object Object]

• API function: The LayerNorm operator is a common normalization operation used in large models. The LayerNormQuant operator integrates the output of the LayerNorm normalization with the downstream quantization operator to reduce the data transfer operations.
• Formula:

  • LayerNorm operation:

    $$y = {{x-E(x)}\over\sqrt {Var(x)+epsilon}} * gamma + beta$$ $$E(x) = {\frac{1}{n} \sum_{i=1}^{n} x_i }$$ $$Var(x) = {\frac{1}{n} \sum_{i=1}^{n} (x_i-E(x))^2 }$$

  • When quantMode is set to 0, the quantization mode is static quantization, and the output scaleOut is meaningless.

    $$res = y / scale + zeroPointsOptional$$

  • When quantMode is set to 1, the quantization mode is dynamic quantization.

    $$tmp = y * scale$$ $$scaleOut = row\_max(abs(tmp))/dtypeMax$$ $$res = round(y / scaleOut )$$

[object Object]

Each operator has calls. First, [object Object] is called to obtain the input parameters and compute the required workspace size based on the process. Then, [object Object] is called to perform computation.

[object Object]

• Parameters

  [object Object]

• Returns

  [object Object]: status code. For details, see .

  The first-phase API implements input parameter validation. The following error codes may be returned.

  [object Object]

[object Object]

• Parameters

  [object Object]

• Returns

  [object Object]: status code. For details, see .

[object Object]

• Deterministic computation:
  • The aclnnLayerNormQuant is implemented in deterministic mode by default.

[object Object]

The following example is for reference only. For details, see .