[object Object]

[object Object][object Object]undefined[object Object]

• Description: Performs quantized group matrix computation, adds the matrix computation results of all groups, and outputs the result.

• Formula:

$$out = \sum_{i=0}^{batch}(x1_i @ x2_i) * x1Scale * x2Scale$$ [object Object]

Each operator has calls. First, aclnnQuantMatmulReduceSumWeightNzGetWorkspaceSize is called to obtain the workspace size required for computation and the executor that contains the operator computation process. Then, aclnnQuantMatmulReduceSumWeightNz is called to perform computation.

• [object Object]

• [object Object]

[object Object]

• Parameters:

  • x1 (aclTensor*, compute input): input x1 in the formula, aclTensor on the device. The can be ND. are not supported. The shape can be three-dimensional (batch, m, k). The data type can be INT8.

  • x2 (aclTensor*, compute input): input x2 in the formula, aclTensor on the device. The data type can be INT8. are not supported. The supports the AI processor affinity data layout format. The shape can be five-dimensional.

    • When transposeX2 is false, the shape is represented by (batch, n1, k1, k0, n0), where k0 = 16 and n0 = 32. k in the shape of x1 and k1 in the shape of x2 must meet the following relationship: ceil(k/16) = k1. n1 in the shape of x2 and n in the shape of out must meet the following relationship: ceil(n/n0) = n1.
    • aclnnCalculateMatmulWeightSizeV2 and aclnnTransMatmulWeight can be used to convert the input format from ND to AI processor affinity format. The original shape in ND format is (batch, k, n).

  • x1Scale (aclTensor*, compute input): input x1Scale in the formula, aclTensor on the device. The can be ND. The shape can be two-dimensional (batch, m). The data type can be FLOAT32.

    • During actual computation, x1Scale is broadcast as (batch, m, n).

  • x2Scale (aclTensor*, compute input): quantization parameter, x2Scale in the formula, aclTensor on the device. The can be ND. The shape can be one-dimensional (n,), where n is the same as that of x2. The data type can be BFLOAT16.

    • During actual computation, x2Scale is broadcast as (batch, m, n).

  • yScale (aclTensor*, compute input): reserved parameter. This parameter is not supported in the current version. It must be set to nullptr.

  • x1Offset (aclTensor*, compute input): reserved parameter. This parameter is not supported in the current version. It must be set to nullptr.

  • x2Offset (aclTensor*, compute input): reserved parameter. This parameter is not supported in the current version. It must be set to nullptr.

  • yOffset (aclTensor*, compute input): reserved parameter. This parameter is not supported in the current version. It must be set to nullptr.

  • bias (aclTensor*, compute input): reserved parameter. This parameter is not supported in the current version. It must be set to nullptr.

  • transposeX1 (bool, compute input): whether the input shape of x1 is transposed. In the current version, only false is supported, indicating that the meaning of the input shape of x1 remains unchanged.

  • transposeX2 (bool, compute input): whether the input shape of x2 is transposed. In the current version, only false is supported, indicating that the meaning of the input shape of x2 remains unchanged.

  • groupSize (int64_t, compute input): reserved parameter. This parameter is not supported in the current version. It must be set to 0.

  • dims (aclIntArray *): aclIntArray on the host, which specifies the reduce dimension. The data type can be INT64. In the current version, only [0] is supported, indicating that ReduceSum is performed on the 0th dimension (batch dimension).

  • keepDims (bool, compute input): whether to retain the dimensions of the input tensor in the output tensor. In the current version, only false is supported.

  • out (aclTensor*, compute output): output out in the formula, aclTensor on the device. The can be ND. are supported. The data type can be BFLOAT16. The shape can be two-dimensional (m, n).

  • workspaceSize (uint64_t*, output): size of the workspace to be allocated on the device.

  • executor (aclOpExecutor**, output): operator executor, containing the operator computation process.

• Returns:

  aclnnStatus: status code. For details, see .

[object Object]

• Parameters:

  • workspace (void *, input): address of the workspace to be allocated on the device.

  • workspaceSize (uint64_t, input): size of the workspace to be allocated on the device, which is obtained by the first-phase API aclnnQuantMatmulReduceSumWeightNzGetWorkspaceSize.

  • executor (aclOpExecutor*, input): operator executor, containing the operator computation process.

  • stream (aclrtStream, input): stream for executing the task.

• Returns:

  aclnnStatus: status code. For details, see .

[object Object]

• Deterministic description:
  • [object Object]Atlas training series products[object Object] and [object Object]Atlas inference series products[object Object]: aclnnQuantMatmulReduceSumWeightNz defaults to a non-deterministic implementation. You can call aclrtCtxSetSysParamOpt to enable deterministic compute.

The input and output support the following data type combinations:

[object Object]undefined[object Object]

The following example is for reference only. For details, see .