[object Object]

[object Object][object Object]undefined[object Object]

• Description: Performs per-token symmetric dynamic quantization on the input tensor.

• Formula:

  • When smoothScalesOptional is not provided:
  $$scaleOut=row\_max(abs(x))/dtypeMax$$ $$yOut=round(x/scaleOut)$$
  • When smoothScalesOptional is provided:
  $$input = x\cdot smoothScalesOptional$$ $$scaleOut=row\_max(abs(input))/dtypeMax$$ $$yOut=round(input/scaleOut)$$

  Where row_max denotes taking the maximum value per row, and dtypeMax is the maximum value of the output data type.

[object Object]

Each operator has calls. First, aclnnDynamicQuantGetWorkspaceSize is called to obtain the workspace size required for computation and the executor that contains the operator computation flow. Then, aclnnDynamicQuant is called to perform computation.

[object Object]

• Parameters

  [object Object]

  • [object Object]Atlas A2 training products/Atlas A2 inference products[object Object] and [object Object]Atlas A3 training products/Atlas A3 inference products[object Object]:

    The data type of the output parameter [object Object] can only be INT4, INT8, or INT32.

  • [object Object]Atlas inference products[object Object] and [object Object]Atlas training products[object Object]:

    • The data types of [object Object] and [object Object] must be FLOAT16.
    • The data type of [object Object] must be INT8.
    • The input parameter [object Object] is reserved and is not involved in the calculation in the current version.

• Returns

  [object Object]: status code. For details, see .

  The first-phase API implements input parameter verification. The following errors may be thrown.

  [object Object]

[object Object]

• Parameters

  [object Object]

• Returns

  [object Object]: status code. For details, see .

[object Object]

• [object Object]Atlas A2 training products/Atlas A2 inference products[object Object], [object Object]Atlas A3 training products/Atlas A3 inference products[object Object], and Ascend 950PR/Ascend 950DT:
  • When the data type of yOut is INT4, the last dimensions of both x and yOut must be divisible by 2.
  • When the data type of yOut is INT32, the last dimension of x must be divisible by 8.
• [object Object]Atlas inference products[object Object] and [object Object]Atlas training products[object Object]: The last dimension of the input x must be exactly divisible by 32. Currently, only symmetric quantization is supported. The BFLOAT16 data type is not supported.
• Deterministic computation:
  • aclnnDynamicQuant defaults to a deterministic implementation.

[object Object]

The following example is for reference only. For details, see .