[object Object]

[object Object][object Object]undefined[object Object]

• Description: Fuses AdaLayerNorm with downstream quantization (only DynamicQuant is supported currently). This operator is used to perform quantization operations on adaptive layer normalization, that is, to normalize the input data and quantize it into low-precision integers to improve the compute efficiency and reduce the memory usage.

• Formula:

  1. Perform LayerNorm normalization on the input x.
  $$LayerNorm(x) = {{x-E(x)}\over\sqrt {Var(x)+epsilon}} * weightOptional + biasOptional$$
  2. Adjust the normalization result using the adaptive parameters scale and shift.
  $$y = LayerNorm(x) * (1 + scale) + shift$$
  3. If smoothScalesOptional is not empty, then:
  $$y = y \cdot smoothScalesOptional$$
  4. Calculate the maximum absolute value of y and divide the value by 127 to calculate the quantization factor that needs to be quantized into the INT8 format.
  $$quantScale = row\_max(abs(y)) / 127$$
  5. Finally, divide y by the quantization factor and round the quotient off to obtain the quantized output.
  $$out = round(y / quantScale)$$

  E(x) indicates the input's mean value, Var(x) indicates the input's variance, and row_max indicates the maximum value of each row.

[object Object]

Each operator has calls. First, aclnnAdaLayerNormQuantGetWorkspaceSize is called to obtain the workspace size required for computation and the executor that contains the operator computation process. Then, aclnnAdaLayerNormQuant is called to perform computation.

[object Object]

• Parameters:

  [object Object]

• Returns:

  aclnnStatus: status code. For details, see .

  The first-phase API implements input parameter verification. The following errors may be thrown.

  [object Object]

[object Object]

• Parameters:

  [object Object]

• Returns:

  aclnnStatus: status code. For details, see .

[object Object]

• Deterministic compute:
  • aclnnAdaLayerNormQuant defaults to a deterministic implementation.

[object Object]

The following example is for reference only. For details, see .