[object Object]

[object Object][object Object]undefined[object Object]

• API description: performs MX quantization with the destination data type being FLOAT4. Only the last axis is quantized. All the previous axes are fused together. The input is divided into multiple data blocks based on the given level0BlockSize, and level-1 quantization is performed on each data block to output the quantization scale level0ScaleOut. Then, the level-1 quantization result is used as the new input, and is divided into multiple data blocks based on the given level1BlockSize. Level-2 quantization is performed on each data block to output the quantization scale level1ScaleOut. The data type is converted based on the round_mode to obtain the quantization result yOut. For details, see the figure (see../figures/DynamicDualLevelMxQuant.png).

• Formulas:

  • The input x is grouped into $k_0$ = level0BlockSize groups along the last axis. A group of $k_0$ elements $\{\{x_i\}_{i=1}^{k_0}\}$ is dynamically quantized into $\{level0Scale, \{temp_i\}_{i=1}^{k_0}\}$, where $k_0$ = level0BlockSize. Then, the temp is grouped into $k_1$ = level1BlockSize groups along the last axis. A group of $k_1$ elements $\{\{temp_i\}_{i=1}^{k_1}\}$ is dynamically quantized into $\{level1Scale, \{y_i\}_{i=1}^{k_1}\}$, where $k_1$ = level1BlockSize.
  $$input\_max_i = max_i(abs(x_i))$$ $$level0Scale = input\_max_i / (FP4\_E2M1\_MAX)$$ $$temp_i = cast\_to\_x\_type(x_i / level0Scale), \space i\space from\space 1\space to\space level0BlockSize$$ $$shared\_exp = floor(log_2(max_i(|temp_i|))) - emax$$ $$level1Scale = 2^{shared\_exp}$$ $$y_i = cast\_to\_FP4\_E2M1(temp_i/level1Scale, round\_mode), \space i\space from\space 1\space to\space level1BlockSize$$

  • ​The quantized $y_{i}$ forms the output yOut based on the corresponding positions of $x_{i}$. The level0Scale forms the output level0ScaleOut based on the corresponding groups of the last axis. The level1Scale forms the output level1ScaleOut based on the corresponding groups of the last axis.

  • max_i indicates the maximum value in the ith group.

  • emax: exponent bit of the maximum regular number of the corresponding data type.

    [object Object]undefined

[object Object]

Each operator has [object Object]two-phase API calls[object Object]. You must call aclnnDynamicDualLevelMxQuantGetWorkspaceSize to obtain the workspace size required for computation and the executor that contains the operator computation process, and then call aclnnDynamicDualLevelMxQuant to perform the computation.

[object Object]

• Parameters

  [object Object]

• Returns

  [object Object]: status code. For details, see .

  The first-phase API implements input parameter validation. The following error codes may be returned.

  [object Object]

[object Object]

• Parameters

  [object Object]

• Returns

  [object Object]: status code. For details, see .

[object Object]

• The restrictions on the shapes of x, level0ScaleOut, and level1ScaleOut are as follows:
  • rank(level1ScaleOut) = rank(x) + 1.
  • level0ScaleOut.shape[-1] = ceil(x.shape[-1] / level0Blocksize).
  • level1ScaleOut.shape[-2] = (ceil(x.shape[-1] / level1Blocksize) + 2 - 1) / 2.
  • level1ScaleOut.shape[-1] = 2.
  • The shapes of other dimensions are consistent with those of input x.
• Deterministic description: The aclnnDynamicDualLevelMxQuant is implemented in deterministic mode by default.

[object Object]

The following example is for reference only. For details, see .