[object Object]

Note: This API will be deprecated in later versions. Use the latest aclnnQuantMatmulV5 API instead.

[object Object][object Object]undefined[object Object]

• Description: Performs quantized matrix multiplication, supporting at least two-dimensional input and at most six-dimensional input. Similar APIs include aclnnMm (only two-dimensional tensors can be used as the input of matrix multiplication) and aclnnBatchMatMul (only three-dimensional matrix multiplication is supported, whose first dimension is the batch dimension). It supports the T-C && T-T .
• Formula:

  • No bias:

    $$out = x1@x2 * scale + offset$$

  • bias (INT32):

    $$out = (x1@x2 + bias) * scale + offset$$

  • [object Object]Atlas A2 training products/Atlas A2 inference products[object Object], [object Object]Atlas A3 training products/Atlas A3 inference products[object Object], and Ascend 950PR/Ascend 950DT: bias BFLOAT16/FLOAT32 is supported (no offset in this scenario).

    $$out = x1@x2 * scale + bias$$

[object Object]

Each operator has calls. First, aclnnQuantMatmulV3GetWorkspaceSize is called to obtain the workspace size required for computation and the executor that contains the operator computation process. Then, aclnnQuantMatmulV3 is called to perform computation.

[object Object]

• Parameters:

  [object Object]
  • [object Object]Atlas inference products[object Object]:
    • x1 and x2 support INT8.
    • scale supports UINT64 and INT64.
    • bias supports INT32.
    • out supports FLOAT16 and INT8.
  • [object Object]Atlas A2 training products/Atlas A2 inference products[object Object] and [object Object]Atlas A3 training products/Atlas A3 inference products[object Object]:
    • x1 and x2 support INT8, INT32, and INT4.
    • scale supports UINT64, INT64, FLOAT32, and BFLOAT16.
    • bias supports INT32, BFLOAT16, and FLOAT32. When x1 and x2 are of type INT32 or INT4, the shape of bias supports only one dimension (n,).
    • When x1 and x2 are of type INT32 or INT4, transposeX1 can only be false.
    • out supports FLOAT16, INT8, BFLOAT16, and INT32.
  • Ascend 950PR/Ascend 950DT:
    • x1 and x2 support INT8.
    • scale supports UINT64, INT64, FLOAT32, and BFLOAT16.
    • scale supports INT32, BFLOAT16, and FLOAT32.
    • out supports FLOAT16, INT8, BFLOAT16, and INT32.
    • When x2 is in ND format, if x1 is an empty tensor with m = 0 or x2 is an empty tensor with n = 0, the output is an empty tensor. When x2 is in FRACTAL_NZ format, if x1 is an empty tensor with m = 0, the output is an empty tensor.

• Returns:

  [object Object]: status code. For details, see .

  The first phase API verifies the input parameters. An error is reported in the following scenarios:

  [object Object]

[object Object]

• Parameters:

  [object Object]

• Returns:

  [object Object]: status code. For details, see .

[object Object]

• Deterministic description:

  • [object Object]Atlas training products[object Object] and [object Object]Atlas inference products[object Object]: aclnnQuantMatmulV3 defaults to a deterministic implementation.
  • Ascend 950PR/Ascend 950DT: The aclnnQuantMatmulV3 function is implemented in a deterministic manner by default.

• [object Object]Atlas inference products[object Object]:

  • The size of the last dimension of x1 cannot exceed 65535. The last dimension of x1 refers to m when transposeX1 is true or k when transposeX1 is false.
  • The size of the last dimension of x2 cannot exceed 65535. The last dimension of x2 refers to k when transposeX2 is true or n when transposeX2 is false.
    • When x2 is in NZ format, transposeX2 cannot be false.
  • Before calling this API, you can use to process x2 whose format is ND to obtain the NZ format.

• [object Object]Atlas A2 training products/Atlas A2 inference products[object Object] and [object Object]Atlas A3 training products/Atlas A3 inference products[object Object]:

  • The size of the last dimension of x1 cannot exceed 65535. The last dimension of x1 refers to m when transposeX1 is true or k when transposeX1 is false. When the data type of x1 is INT32 or INT4, the INT4 quantization scenario is used. Currently, only the scenario where transposeX1 is false is supported. When the data type of x1 is INT4, the shape is represented by (batch, m, k), where k must be an even number. When the data type of x1 is INT32, each INT32 data entry stores eight INT4 data entries, with shape represented by (batch, m, k // 8), where k must be a multiple of 8.
  • The size of the last dimension of x2 cannot exceed 65535. The last dimension of x2 refers to k when transposeX2 is true or n when transposeX2 is false. When x2 is in NZ format, transposeX2 cannot be false.
    • If the data type is INT4 and transposeX2 is true, the shape is (n, k), and k must be an even number. If transposeX2 is false, the shape is (k, n), and n must be an even number.
    • If the data type is INT32, eight INT4 data elements are stored in each INT32 data element. When transposeX2 is true, the shape is (n, k // 8), and k must be a multiple of 8. When transposeX2 is false, the shape is (k, n // 8), and n must be a multiple of 8.
    • The aclnnConvertWeightToINT4Pack API can be used to convert x2 from INT32 (one int32 space stores one int4 data entry in bits 0–3) to INT32 (one int32 space stores eight int4 data entries) or INT4 (one int4 space stores one int4 data entry). For details, see aclnnConvertWeightToINT4Pack.
  • Before this API is called, you can call the API to convert the format of x2 from ND to NZ.

• Ascend 950PR/Ascend 950DT:

  • When the last two axes are 1 (that is, n = 1 or k = 1), x2 does not support the proprietary format and supports only the ND format.
  • Before this API is called, you can call the or API to convert the format of x2 from ND to NZ.
  • If the last two dimensions of the original ND shape are 1, you are not advised to convert the format to NZ. By default, x2 is discontinuous, and only discontinuous x2 tensors are supported.

The following data type combinations are supported for the input and output. The combinations support T-C && T-T .

[object Object]

• [object Object]Atlas inference products[object Object]:

  [object Object]undefined

• [object Object]Atlas A2 training products/Atlas A2 inference products[object Object] and [object Object]Atlas A3 training products/Atlas A3 inference products[object Object]:

  [object Object]undefined

• Ascend 950PR/Ascend 950DT:

  [object Object]undefined

[object Object]

• For the [object Object]Atlas A2 training products/Atlas A2 inference products[object Object], [object Object]Atlas A3 training products/Atlas A3 inference products[object Object], and Ascend 950PR/Ascend 950DT: The following is a code example in general scenarios, which is for reference only. For details about the compilation and execution process, see .

  [object Object]

• For the [object Object]Atlas A2 training products/Atlas A2 inference products[object Object] and [object Object]Atlas A3 training products/Atlas A3 inference products[object Object], the sample code in the NZ scenario (transposeX2=false) is as follows, which is for reference only. For details about the compilation and execution process, see .

  [object Object]

• For the [object Object]Atlas A2 training products/Atlas A2 inference products[object Object] and [object Object]Atlas A3 training products/Atlas A3 inference products[object Object], the sample code for INT4 quantization is as follows (x1 and x2 are of the INT4 type, and transposeX2 is false). This is for reference only. For details about the compilation and execution process, see .

  [object Object]

• [object Object]Atlas inference products[object Object]: The sample code (transposeX2 = true) when x2 is in AI processor affinity data layout format is as follows (for reference only). For details about the compilation and running process, see .

  [object Object]