[object Object]

[object Object][object Object]undefined[object Object]

• Function: Dynamically quantizes the input tensor. In the MOE scenario, smoothScalesOptional for each expert is different and is distinguished by the input groupIndexOptional. Supports symmetric and asymmetric quantization. Supports per-token, per-tensor, and per-channel quantization modes. Compared with aclnnDynamicQuantV2, the per-tensor and per-channel quantization modes are added and specified by the quantMode parameter.

• Formulas:

  • Symmetric quantization:
    • When smoothScalesOptional is not provided:$$scaleOut=\max_{t}(abs(x))/DTYPE_{MAX}$$ $$yOut=round(x/scaleOut)$$
    • When smoothScalesOptional is provided:$$input = x\cdot smoothScalesOptional$$ $$scaleOut=\max_{t}(abs(input))/DTYPE_{MAX}$$ $$yOut=round(input/scaleOut)$$
  • Asymmetric quantization:
    • When smoothScalesOptional is not provided:$$scaleOut=(\max_{t}(x) - \min_{t}(x))/(DTYPE_{MAX} - DTYPE_{MIN})$$ $$offset=DTYPE_{MAX}-\max_{t}(x)/scaleOut$$ $$yOut=round(x/scaleOut+offset)$$
    • When smoothScalesOptional is provided:$$input = x\cdot smoothScalesOptional$$ $$scaleOut=(\max_{t}(input) - \min_{t}(input))/(DTYPE_{MAX} - DTYPE_{MIN})$$ $$offset=DTYPE_{MAX}-\max_{t}(input)/scaleOut$$ $$yOut=round(input/scaleOut+offset)$$ $\max_{t}$/$\min_{t}$ indicates the mode of calculating the maximum or minimum value. If quantMode is set to pertoken, t is set to row, indicating that the maximum or minimum value is calculated for each token. If quantMode is set to pertensor, t is set to all, indicating that the maximum or minimum value is calculated for the entire tensor. If quantMode is set to perchannel, t is set to col, indicating that the maximum or minimum value is calculated for each channel. $DTYPE_{MAX}$ is the maximum value of the output type, and $DTYPE_{MIN}$ is the minimum value of the output type.

[object Object]

Each operator has [object Object]two-phase API calls[object Object]. You must call aclnnDynamicQuantV3GetWorkspaceSize to obtain the workspace size required for computation and the executor that contains the operator computation process, and then call aclnnDynamicQuantV3 to perform the computation.

[object Object]

• Parameters

  [object Object]

• Returns

  [object Object]: status code. For details, see .

  The first-phase API implements input parameter validation. The following error codes may be returned.

  [object Object]

[object Object]

• Parameters

  [object Object]

• Returns

  [object Object]: status code. For details, see .

[object Object]

• Deterministic computation:

  • The default deterministic implementation of aclnnDynamicQuantV3 is used.

  When the data type of yOut is INT4, the last dimensions of x and yOut must be divisible by 2. When the data type of yOut is INT32, the last dimension of x must be divisible by 8. When groupIndexOptional is specified, the number of experts cannot exceed the product of the dimensions of x excluding the last dimension. The value of groupIndexOptional must be a non-decreasing array of non-negative integers, and the last value must be equal to the product of the dimensions of x excluding the last dimension. If this condition is not met, the result is meaningless.

[object Object]

The following example is for reference only. For details, see .