[object Object]

[object Object][object Object]undefined[object Object]

• Description: Performs matrix multiplication for quantization. Similar APIs include aclnnMm (only two-dimensional tensors can be used as the input of matrix multiplication) and aclnnBatchMatMul (only three-dimensional matrix multiplication is supported, whose first dimension is the batch dimension). It supports T-C, T-T, K-C, and K-T .

• Formula:

  • No x1Scale, no bias:
  $$out = x1@x2 * x2Scale + x2Offset$$
  • bias INT32:
  $$out = (x1@x2 + bias) * x2Scale + x2Offset$$
  • bias BFLOAT16/FLOAT32 (no x2Offset in this scenario):
  $$out = x1@x2 * x2Scale + bias$$
  • With x1Scale, no bias:
  $$out = x1@x2 * x2Scale * x1Scale$$
  • With x1Scale, bias INT32 (no x2Offset in this scenario):
  $$out = (x1@x2 + bias) * x2Scale * x1Scale$$
  • With x1Scale, bias BFLOAT16/FLOAT16/FLOAT32 (no x2Offset in this scenario):
  $$out = x1@x2 * x2Scale * x1Scale + bias$$

[object Object]

Each operator has calls. First, aclnnQuantMatmulWeightNzGetWorkspaceSize is called to obtain the workspace size required for computation and the executor that contains the operator computation process. Then, aclnnQuantMatmulWeightNz is called to perform computation.

• [object Object]

• [object Object]

[object Object]

• Parameters:

  • x1 (aclTensor*, compute input): input x1 in the formula, aclTensor on the device. The data type can be INT8. Non-contiguous tensors are supported only when the last two axes are transposed. are not supported in other scenarios. The can be ND. The shape supports two to six dimensions.
    • When transposeX1 is false, the shape is (batch, m, k), where batch is optional.
    • When transposeX1 is true, the shape is (batch, k, m), where batch is optional.
  • x2 (aclTensor*, compute input): input x2 in the formula, aclTensor on the device. The data type can be INT8. The supports the AI processor affinity data layout format. The shape supports four to eight dimensions.
    • When transposeX2 is true, the shape is represented by (batch, k1, n1, n0, k0), where batch is optional, k0 = 32, and n0 = 16. k in the shape of x1 and k1 in the shape of x2 must meet the following relationship: ceil(k/32) = k1. n1 in the shape of x2 and n in the shape of out must meet the following relationship: ceil(n/n0) = n1.
    • When transposeX2 is false, the shape is represented by (batch, n1, k1, k0, n0), where batch is optional, k0 = 16, and n0 = 32. k in the shape of x1 and k1 in the shape of x2 must meet the following relationship: ceil(k/16) = k1. n1 in the shape of x2 and n in the shape of out must meet the following relationship: ceil(n/n0) = n1.
    • aclnnCalculateMatmulWeightSizeV2 and aclnnTransMatmulWeight can be used to convert the input format from ND to AI processor affinity format.
    • [object Object]Atlas A2 training products/Atlas A2 inference products[object Object], [object Object]Atlas A3 training series products/Atlas A3 inference series products[object Object], and [object Object]Atlas inference series products[object Object]: Non-contiguous tensors are not supported.
  • x1Scale (aclTensor*, compute input): input x1Scale in the formula, aclTensor on the device. It is an optional quantization parameter.
    • [object Object]Atlas A2 training products/Atlas A2 inference products[object Object] and [object Object]Atlas A3 training series products/Atlas A3 inference series products[object Object]: The data type can be FLOAT32. The can be ND. The shape is one-dimensional (t,), where t = m and m is the same as that of x1.
    • [object Object]Atlas inference series products[object Object]: x1Scale is not supported.
  • x2Scale (aclTensor*, compute input): input x2Scale in the formula, aclTensor on the device. It is a quantization parameter. The can be ND. The shape is one-dimensional (t,), where t = 1 or n, and n is the same as that of x2.
    • [object Object]Atlas A2 training products/Atlas A2 inference products[object Object] and [object Object]Atlas A3 training series products/Atlas A3 inference series products[object Object]: The data type can be UINT64, INT64, FLOAT32, or BFLOAT16.
    • [object Object]Atlas inference series products[object Object]: The data type can be UINT64 or INT64.
    • If the original input type does not conform with the combinations described in , call the aclnn API of the TransQuantParamV2 operator to convert scale to the INT64 or UINT64 type in advance.
  • yScale (aclTensor*, compute input): reserved parameter. This parameter is not supported in the current version. It must be set to a nullptr or empty tensor.
  • x1Offset (aclTensor*, compute input): reserved parameter. This parameter is not supported in the current version. It must be set to a nullptr or empty tensor.
  • x2Offset (aclTensor*, compute input): input x2Offset in the formula, aclTensor on the device. It is an optional quantization parameter. The data type can be FLOAT32. The can be ND. The shape is one-dimensional (t,), where t = 1 or n, and n is the same as that of x2.
  • yOffset (aclTensor*, compute input): reserved parameter. This parameter is not supported in the current version. It must be set to a nullptr or empty tensor.
  • bias (aclTensor*, compute input): input bias in the formula, aclTensor on the device. This parameter is optional. The can be ND. The shape can be one-dimensional (n,) or three-dimensional (batch, 1, n), where n is the same as that of x2. When the shape of out is two-, four-, five-, or six-dimensional, the shape of bias can only be one-dimensional (n,).
    • [object Object]Atlas A2 training products/Atlas A2 inference products[object Object] and [object Object]Atlas A3 training series products/Atlas A3 inference series products[object Object]: The data type can be INT32, BFLOAT16, FLOAT16, or FLOAT32.
    • [object Object]Atlas inference series products[object Object]: The data type can be INT32.
  • transposeX1 (bool, compute input): whether the input shape of x1 is transposed. If transposeX1 is false, the shape is (batch, m, k). If transposeX1 is true, the shape is (batch, k, m), where batch is optional.
  • transposeX2 (bool, compute input): whether the input shape of x2 is transposed. If transposeX2 is true, the shape is (batch, k1, n1, n0, k0), where batch is optional, k0 = 32, and n0 = 16. k in the shape of x1 and k1 in the shape of x2 must meet the following relationship: ceil(k/32) = k1. If transposeX2 is false, the shape is represented by (batch, n1, k1, k0, n0), where batch is optional, k0 = 16, and n0 = 32. k in the shape of x1 and k1 in the shape of x2 must meet the following relationship: ceil(k/16) = k1.
  • groupSize (int64_t, compute input): reserved parameter. This parameter is not supported in the current version. It must be set to 0.
  • out (aclTensor*, compute output): output out in the formula, aclTensor on the device. are supported. The can be ND. The shape supports two to six dimensions (batch, m, n), where batch is optional. The batch dimensions of x1 and x2 can be broadcast. The output batch is the same as the broadcast batch. m is the same as m of x1. n meets the relationship ceil(n/n0) = n1 with n1 and n0 of x2.
    • [object Object]Atlas A2 training products/Atlas A2 inference products[object Object] and [object Object]Atlas A3 training series products/Atlas A3 inference series products[object Object]: The data type can be FLOAT16, INT8, BFLOAT16, or INT32.
    • [object Object]Atlas inference series products[object Object]: The data type can be FLOAT16 or INT8.
  • workspaceSize (uint64_t*, output): size of the workspace to be allocated on the device.
  • executor (aclOpExecutor**, output): operator executor, containing the operator computation process.

• Returns:

  aclnnStatus: status code. For details, see .

[object Object]

• Parameters:

  • workspace (void*, input): address of the workspace to be allocated on the device.
  • workspaceSize (uint64_t, input): size of the workspace to be allocated on the device, which is obtained by the first-phase API aclnnQuantMatmulWeightNzGetWorkspaceSize.
  • executor (aclOpExecutor*, input): operator executor, containing the operator computation process.
  • stream (aclrtStream, input): stream for executing the task.

• Returns:

  aclnnStatus: status code. For details, see .

[object Object]

• Deterministic description:

  • [object Object]Atlas training series products[object Object] and [object Object]Atlas inference series products[object Object]: aclnnQuantMatmulWeightNz defaults to a deterministic implementation.

• [object Object]Atlas A2 training products/Atlas A2 inference products[object Object], [object Object]Atlas A3 training series products/Atlas A3 inference series products[object Object], and [object Object]Atlas inference series products[object Object]: Before calling this API, you can use to process x2 in ND format to obtain x2 in AI processor affinity format. The input and output support the following data type combinations:

• [object Object]Atlas A2 training products/Atlas A2 inference products[object Object] and [object Object]Atlas A3 training series products/Atlas A3 inference series products[object Object]:

  [object Object]undefined

• [object Object]Atlas inference series products[object Object]:

  [object Object]undefined

The following data type combinations support T-C && T-T when x1Scale is null: When x1Scale is not null, K-C && K-T is supported.

[object Object]undefined[object Object]

• [object Object]Atlas A2 training products/Atlas A2 inference products[object Object] and [object Object]Atlas A3 training series products/Atlas A3 inference series products[object Object]: The sample code (transposeX2=false) when x2 is in AI processor affinity format is as follows (for reference only). For details about the compilation and running process, see .

  [object Object]

• [object Object]Atlas inference series products[object Object]: The sample code (transposeX2=true) when x2 is in AI processor affinity format is as follows (for reference only). For details about the compilation and running process, see .

  [object Object]