[object Object]

[object Object][object Object]undefined[object Object]

• Description: Performs per-token symmetric dynamic quantization on the input tensor.

• Formula:

  • When smoothScalesOptional is not provided:
  $$scaleOut=row\_max(abs(x))/dtypeMax$$ $$yOut=round(x/scaleOut)$$
  • When smoothScalesOptional is provided:
  $$input = x\cdot smoothScalesOptional$$ $$scaleOut=row\_max(abs(input))/dtypeMax$$ $$yOut=round(input/scaleOut)$$

  Where row_max denotes taking the maximum value per row, and dtypeMax is the maximum value of the output data type.

[object Object]

Each operator has calls. First, aclnnDynamicQuantGetWorkspaceSize is called to obtain the workspace size required for computation and the executor that contains the operator computation flow. Then, aclnnDynamicQuant is called to perform computation.

[object Object]

• Parameters

  [object Object]

  • [object Object]Atlas A2 training products/Atlas A2 inference products[object Object] and [object Object]Atlas A3 training series products/Atlas A3 inference series products[object Object]:

    The data type of [object Object] must be INT4 or INT8.

  • [object Object]Atlas inference series products[object Object] and [object Object]Atlas training series products[object Object]:

    • The data types of [object Object] and [object Object] must be FLOAT16.
    • The data type of [object Object] must be INT8.

• Returns

  aclnnStatus: status code. For details, see .

  The first-phase API implements input parameter verification. The following errors may be thrown.

  [object Object]

[object Object]

• Parameters

  [object Object]

• Returns

  aclnnStatus: status code. For details, see .

[object Object]

• When the data type of yOut is INT4, the last dimensions of both x and yOut must be divisible by 2.
• When the data type of yOut is INT32, the last dimension of x must be divisible by 8.
• [object Object]Atlas inference series products[object Object]: Only 32-bit aligned data is supported on the tail axis. Currently only symmetric quantization is supported, and the BFLOAT16 data type is not supported.
• Deterministic computation:
  • aclnnDynamicQuant defaults to a deterministic implementation.

[object Object]

The following example is for reference only. For details, see .