[object Object][object Object][object Object]undefined[object Object]

• API function: Quantizes the input x, performs matrix multiplication, and performs dequantization.

• Formula:

  1. If smoothScaleOptional is input, then:
  $$x = x\cdot scale_{smooth}$$
  2. If xScaleOptional is not input, dynamic quantization is performed, and the x quantization coefficient needs to be computed.
  $$scale_{x}=row\_max(abs(x))/max_{quantDataType}$$
  3. Quantization
  $$x_{quantized}=round(x/scale_{x})$$
  4. Matrix multiplication + dequantization

  • 4.1 If the data type of the input $scale_{weight}$ is FLOAT32, then:

    $$out = (x_{quantized}@weight_{quantized} + bias) * scale_{weight} * scale_{x}$$

  • 4.2 If the data type of the input $scale_{weight}$ is INT64, then:

    $$scale_{weight} = torch.tensor(np.frombuffer(scale_{weight}.numpy().astype(np.int32).tobytes(), dtype=np.float32)) \\ out = (x_{quantized}@weight_{quantized} + bias) * scale_{weight}$$

    Note: In the scenario described in 4.2, the matrix multiplication operation has been performed on $scale_{weight}$ and $scale_{x}$ before the input of $scale_{weight}$. Therefore, this step is omitted during internal operator computation. This requires that the scenario must be pertensor static quantization. That is, before input, $scale_{weight}$ must be processed as follows to obtain data of the INT64 type:

    $$scale_{weight} = scale_{weight} * scale_{x} \\ scale_{weight} = torch.tensor(np.frombuffer(scale_{weight}.numpy().astype(np.float32). \\tobytes(), dtype=np.int32).astype(np.int64))$$

[object Object]

Each operator has calls. First, aclnnQuantMatmulDequantGetWorkspaceSize is called to obtain the input parameters and compute the required workspace size based on the process. Then, aclnnQuantMatmulDequant is called to perform computation.

[object Object]

• Parameters

  [object Object]
  • When the weight parameter is in ND format, the shape is two-dimensional.
    • When transposeWeight is set to true, the shape is represented by (n, k).
    • When transposeWeight is set to false, the shape is represented by (k, n).
  • When the weight parameter is in FRACTAL_NZ format, the shape can be four-dimensional.
    • When transposeWeight is set to true, the shape is represented by (k1, n1, n0, k0), where k0 = 32, n0 = 16, and k1 and k of x must meet the following relationship: ceilDiv (k, 32) = k1.
    • When transposeWeight is set to false, the shape is represented by (n1, k1, k0, n0), where k0 = 16, n0 = 32, and k1 and k of x must meet the following relationship: ceilDiv (k, 16) = k1.
    • aclnnCalculateMatmulWeightSizeV2 and aclnnTransMatmulWeight can be used to convert the input format from ND to FRACTAL_NZ.

• Returns

  [object Object]: status code. For details, see .

  The first-phase API implements input parameter verification. The following errors may be thrown:

  [object Object]

[object Object]

• Parameters

  [object Object]

• Returns

  [object Object]: status code. For details, see .

[object Object]

• Deterministic description:

  • [object Object]Atlas training products[object Object] and [object Object]Atlas inference products[object Object]: aclnnQuantMatmulDequant defaults to a deterministic implementation.

• n and k must be integer multiples of 16.

• If the data type of weightScale is INT64, the data type of xScaleOptional must be FLOAT16 and the value of xQuantMode must be pertensor. If the data type of xScaleOptional is FLOAT16, the data type of weightScale must be INT64 and the value of xQuantMode must be pertensor.

[object Object]

The following example is for reference only. For details, see .