[object Object]

[object Object][object Object]undefined[object Object]

• API description: performs MX quantization when the destination data type is FLOAT4 or FLOAT8. On the given axis, the quantization scale mxscale corresponding to the group of numbers is calculated based on the number of elements in each block. The calculated mxscale is used as the corresponding part of the output mxscaleOut. Then, each number in the group is divided by mxscale and converted to the corresponding dstType based on round_mode, and the obtained quantization result y is used as the corresponding part of the output yOut. When dstType is FLOAT8_E4M3FN or FLOAT8_E5M2, the algorithm for calculating mxscale is specified based on the value of scaleAlg.

• Formulas:

  • Scenario 1: When scaleAlg is 0:

    • The input x is grouped by k = blocksize in the axis dimension. A group of k numbers $\{\{V_i\}_{i=1}^{k}\}$ is dynamically quantized to $\{mxscale1, \{P_i\}_{i=1}^{k}\}$, where k = blocksize.
    $$shared\_exp = floor(log_2(max_i(|V_i|))) - emax \\ mxscale = 2^{shared\_exp}\\ P_i = cast\_to\_dst\_type(V_i/mxscale, round\_mode), \space i\space from\space 1\space to\space blocksize\\$$

    • ​The quantized $P_{i}$ forms the output yOut according to the positions of the corresponding $V_{i}$, and mxscale forms the output mxscaleOut according to the groups in the corresponding axis dimension.

    • emax: exponent bit of the maximum regular number of the corresponding data type.

      [object Object]undefined

  • Scenario 2: When scaleAlg is 1, only the FP8 type is involved.

    • A long vector is divided into blocks, each with a length of k. A block scaling factor $S_{fp32}^b$ is calculated for each block, and then all elements in the block are mapped to the target low-precision type FP8 using the same $S_{fp32}^b$. If the last block contains less than k elements, the missing values are considered as 0 and processed as a complete block.
    • Find the maximum absolute value of the numbers in the block:$$Amax(D_{fp32}^b)=max(\{|d_{i}|\}_{i=1}^{k})$$
    • Map the FP32 to the range that can be represented by the target data type FP8. $Amax(DType)$ is the maximum value that can be represented by the target precision.$$S_{fp32}^b = \frac{Amax(D_{fp32}^b)}{Amax(DType)}$$
    • Convert the block scaling factor $S_{fp32}^b$ to the scaling value $S_{ue8m0}^b$ that can be represented in the FP8 format.
    • Extract the unbiased exponent $E_{int}^b$ and mantissa $M_{fixp}^b$ from the floating-point scaling factor $S_{fp32}^b$ of the block.
    • To ensure that no overflow occurs during quantization, the exponent is rounded up and is within the range that can be represented by FP8. $$ E_{int}^b = \begin{cases} E_{int}^b + 1, & \text{ if $S_{fp32}^b$ is a regular number, and $E_{int}^b < 254$ and } M_{fixp}^b > 0 \\ E_{int}^b + 1, & \text{ if $S_{fp32}^b$ is an unnormalized number, and } M_{fixp}^b > 0.5 \\ E_{int}^b, & \text{ otherwise $\end{cases}$ $$
    • Calculate the block scaling factor: $S_{ue8m0}^b=2^{E_{int}^b}$
    • Calculate the block conversion factor: $R_{fp32}^b=\frac{1}{fp32(S_{ue8m0}^b)}$
    • The final step of quantization is to quantize each element in the block. The final output is $\left(S^b, [d^i]_{i=1}^k\right)$, where $d^i = DType(d_{fp32}^i \cdot R_{fp32}^n)$, $S^b$ is the scaling factor of the block ($S_{ue8m0}^b$), and $[d^i]_{i=1}^k$ is the quantized data in the block.

[object Object]

Each operator has [object Object]two-phase API calls[object Object]. You must call aclnnDynamicMxQuantGetWorkspaceSize to obtain the workspace size required for computation and the executor that contains the operator execution process, and then call aclnnDynamicMxQuant to perform the computation.

[object Object]

• Parameters

  [object Object]

• Return Value

  [object Object]: status code. For details, see .

  The first-phase API implements input parameter validation. The following error codes may be returned.

  [object Object]

[object Object]

• Parameters

  [object Object]

• Return Value

  [object Object]: status code. For details, see .

[object Object]

• Deterministic computation:
  • Default deterministic implementation of aclnnDynamicMxQuant.
• The shape constraints on x and mxscaleOut are described as follows:
  • rank(mxscaleOut) = rank(x) + 1.
  • axis_change = axis if axis >= 0 else axis + rank(x).
  • mxscaleOut.shape[axis_change] = (ceil(x.shape[axis] / blocksize) + 2 - 1) / 2.
  • mxscaleOut.shape[-1] = 2.
  • Other dimensions are the same as those of the input x.

[object Object]

The following example is for reference only. For details, see .