[object Object][object Object][object Object]undefined[object Object]

• Description: Performs matrix multiplication for quantization. Similar APIs include aclnnMm (only two-dimensional tensors can be used as the input of matrix multiplication) and aclnnBatchMatMul (only three-dimensional matrix multiplication is supported, whose first dimension is the batch dimension). T-C, T-T, K-C, K-T, and mx quantization modes are supported (../common/quant_mode_introduction.md).

• Formula:

  [object Object]
  • No x1Scale, no bias:
  $$out = x1@x2 * x2Scale + x2Offset$$
  • bias INT32:
  $$out = (x1@x2 + bias) * x2Scale + x2Offset$$
  • With x1Scale and no bias:
  $$out = x1@x2 * x2Scale * x1Scale$$
  • With x1Scale and bias of INT32 type:
  $$out = (x1@x2 + bias) * x2Scale * x1Scale$$ [object Object][object Object]
  • bias BFLOAT16/FLOAT32 (no x2Offset in this scenario):
  $$out = x1@x2 * x2Scale + bias$$
  • With x1Scale, no bias:
  $$out = x1@x2 * x2Scale * x1Scale$$
  • With x1Scale, bias INT32 (no x2Offset in this scenario):
  $$out = (x1@x2 + bias) * x2Scale * x1Scale$$
  • With x1Scale, bias BFLOAT16/FLOAT16/FLOAT32 (no x2Offset in this scenario):
  $$out = x1@x2 * x2Scale * x1Scale + bias$$
  • x1 is INT8, x2 is INT32, x1Scale is FLOAT32, x2Scale is UINT64, and yOffset is FLOAT32:
  $$out = ((x1 @ (x2*x2Scale)) + yOffset) * x1Scale$$ [object Object][object Object]
  • mx quantization mode:$$out[m,n] = \sum_{j=0}^{kLoops-1} ((\sum_{k=0}^{gsK-1} (x1Slice * x2Slice))* (x1Scale[m/gsM, j] * x2Scale[j, n/gsN]))+bias[n]$$ gsM, gsN, and gsK represent groupSizeM, groupSizeN, and groupSizeK, respectively. x1Slice represents the vector of groupSizeK length in row m of x1, and x2Slice represents the vector of groupSizeK length in column n of x2. The K axis is sliced from the start of j*groupSizeK. The value range of j is [0, kLoops], where kLoops = ceil(K / groupSizeK). K indicates the length of the K axis. The length of the last slice can be less than groupSizeK.
  [object Object]

[object Object]

Each operator has calls. First, aclnnQuantMatmulWeightNzGetWorkspaceSize is called to obtain the workspace size required for computation and the executor that contains the operator computation process. Then, aclnnQuantMatmulWeightNz is called to perform computation.

[object Object]

• Parameters:

  [object Object]

  • Formula 1:

    $$groupSize = groupSizeK | groupSizeN << 16 | groupSizeM << 32$$ [object Object][object Object]
    • The superscript "1" in the data type column of the preceding table indicates that the data type is not supported by the series.
    • x2 does not support.
    • yScale is not supported.
    • groupSize is not supported. If groupSize is passed, the value is 0.[object Object]
    [object Object][object Object]
    • The superscript "2" in the "Data Type" column of the table above indicates data types that are not supported by the products. - are not supported.
    • yScale is not supported.
    • groupSize is not supported. If groupSize is set to 0,[object Object]
    [object Object][object Object]
    • The superscript "3" in the data type column of the preceding table indicates that the data type is not supported by the series.
    • x2 supports when the last two axes are transposed. In other scenarios, are not supported.
    • groupSize can be set to a non-zero value.[object Object]

• Returns:

  [object Object]: status code. For details, see .

  The first-phase API implements input parameter verification. The following errors may be thrown:

  [object Object]

[object Object]

• Parameters:

  [object Object]

• Returns:

  [object Object]: status code. For details, see .

[object Object]

• Deterministic description:
  • The default deterministic implementation of aclnnQuantMatmulWeightNz is used.

[object Object]

• Before calling this API, you can use to process x2 whose format is ND to obtain the AI processor affinity data layout format.

• The input and output support the following data type combinations:

  [object Object]undefined

• Restrictions on x1: When the data type is INT8 and the data type of x2 is INT32, transposeX1 must be false. The dimension is (m, k). k must be an even number and less than 29576.

• Restrictions on yOffset: The shape supports one dimension (n). It is the auxiliary result of offline computation during computation. The value must be 8*x2*x2Scale and accumulated in the first dimension.

[object Object][object Object]

• Before calling this API, you can use to process x2 whose format is ND to obtain the AI processor affinity data layout format.

• The input and output support the following data type combinations:

  [object Object]undefined

• When x1Scale is not null, only K-C quantization is supported.

[object Object][object Object]

• Before calling this API, you can use or to process x2 whose format is ND to obtain the NZ format. When using this API, you must fill 0 to prevent dirty data from being introduced.

• If the last two dimensions of the original ND are 1, the weightNz feature cannot be used. This API does not support this scenario.

• Restrictions on the T-C quantization and T-T quantization scenarios:

[object Object][object Object] - The input and output support the following data type combinations: [object Object][object Object]

• Restrictions on K-C and K-T quantization:

[object Object][object Object] - The input and output support the following data type combinations: [object Object][object Object]

• Restrictions on mx quantization:

[object Object][object Object] - The input and output support the following data type combinations: [object Object][object Object]

• In the fake-quantization scenario, the dtype and shape requirements are as follows:

  [object Object]undefined
  • Constraints:
    • The sizes of k and n must be 64-byte aligned.
    • When x1 is of type FLOAT8_E4M3FN and x2 is of type float32, x2 indicates a format in which eight FLOAT4_E2M1 elements are tightly packed into one float32 element.

[object Object][object Object]

The following example is for reference only. For details, see .

• [object Object]Atlas A2 training products/Atlas A2 inference products[object Object] and [object Object]Atlas A3 training products/Atlas A3 inference products[object Object]:

  The following is the sample code for the scenario where x2 is in NZ format (transposeX2=false):.

  [object Object]

• [object Object]Atlas inference products[object Object]: The following is an example of the code in the NZ format (transposeX2=true):

  [object Object]

• Ascend 950PR/Ascend 950DT: The following is an example of the code in the NZ format (transposeX2=true):

  [object Object]