[object Object]

Note: This API will be deprecated in later versions. Use the latest aclnnGroupedMatmulV5 API.

[object Object][object Object]undefined[object Object]

• Description: Implements grouped matrix multiplication, supporting non-uniform matrix dimension sizes across multiple groups. The basic function is matrix multiplication, for example, $y_i[m_i,n_i]=x_i[m_i,k_i] \times weight_i[k_i,n_i], i=1...g$, where $g$ indicates the number of groups and $m_i$, $k_i$, and $n_i$ define the shapes for each group. Both inputs and outputs are of the aclTensorList type, with the following functions:

  • K-axis grouping: $k_i$ varies across groups, while $m_i$ and $n_i$ remain the same for each group. In this case, $x_i$ and $weight_i$ can be concatenated along the K-axis.
  • M-axis grouping: $k_i$ remains the same for each group. In this case, $weight_i$ and $y_i$ can be concatenated along the N-axis.

  Compared with , this API provides the following new features:

  • [object Object]Atlas A2 training products/Atlas A2 inference products[object Object]:
    • Supports weight transposition in non-quantization scenarios. Transposition refers to the case where the shape is [M, K], the stride is [1, M], and the data layout is [K, M].
    • Supports M-axis and K-axis grouping, represented by [object Object].
    • x, weight, and y are all single tensors. In non-quantization scenarios, both x and weight can be of the float32 type.
    • Supports weight transposition and single-tensor weights in quantization and fake-quantization scenarios.
    • For the features supported by , this API does not support scenarios where [object Object] is single-tensor while [object Object] and [object Object] are multi-tensor.
  • Ascend 950PR/Ascend 950DT:
    • In the fake-quantization scenario, weight transpose is supported, and x, weight, and y can be single tensors.
    • For the features supported by , this API does not support scenarios where [object Object] is single-tensor while [object Object] and [object Object] are multi-tensor.

Notes:

• "Single-tensor" means that tensors of all groups in a tensor list are concatenated into one tensor along the axis specified [object Object].

• Tensor transpose: If the tensor shape is [M, K], the stride is [1, M], and the data layout is [K, M], then the tensor is a non-contiguous tensor.

• Formula:

  • Non-quantization scenario:

    $$y_i=x_i\times weight_i + bias_i$$

  • Quantization scenario:

    $$y_i=(x_i\times weight_i + bias_i) * scale_i + offset_i$$

  • Dequantization scenario:

    $$y_i=(x_i\times weight_i + bias_i) * scale_i$$

  • Fake-quantization scenario:

    $$y_i=x_i\times (weight_i + antiquant\_offset_i) * antiquant\_scale_i + bias_i$$

[object Object]

Each operator has calls. First, [object Object] is called to obtain the input parameters and compute the required workspace size based on the process. Then, [object Object] is called to perform computation.

[object Object]

• Parameters

  [object Object]
  • [object Object]Atlas A2 training products/Atlas A2 inference products[object Object]:
    • x supports FLOAT16, BFLOAT16, INT8, and FLOAT32.
    • weight supports FLOAT16, BFLOAT16, INT8, and FLOAT32.
    • biasOptional supports FLOAT16, FLOAT32, and INT32.
    • y supports FLOAT16, BFLOAT16, INT8, and FLOAT32.
    • The input parameters x and weight, and the output parameter y support a maximum of 128 tensors.
  • Ascend 950PR/Ascend 950DT:
    • x supports FLOAT16, BFLOAT16, FLOAT32, and INT8.
    • weight supports FLOAT16, BFLOAT16, FLOAT32, and INT8.
    • biasOptional supports FLOAT16, BFLOAT16, FLOAT32, and INT32.
    • y supports FLOAT16, BFLOAT16, FLOAT32, and INT8.
    • offsetOptional is not supported.
    • groupType supports grouping and non-grouping on the m axis. Only non-quantization supports grouping on the k axis.
    • In the non-quantization scenario, the input parameters x and weight, and the output parameter y support a maximum of 1024 tensors. In the fake-quantization scenario, the input parameters x and weight, and the output parameter y support a maximum of 128 tensors. In the quantization scenario, the input parameters x and weight, and the output parameter y support a maximum of 1 tensor.

• Return

  [object Object]: status code. For details, see .

  The first-phase API implements input parameter validation. The following errors may be thrown.

  [object Object]

[object Object]

• Parameters

  [object Object]

• Return

  [object Object]: status code. For details, see .

[object Object]

• Deterministic computation:

  • [object Object] defaults to deterministic implementation.

• If [object Object] is passed, it must be a non-negative ascending array, and its length cannot be 1.

• The size of each dimension for every tensor in [object Object] and [object Object], after 32-byte alignment, should be less than the maximum value of INT32 (2147483647).

• [object Object]Atlas A2 training products/Atlas A2 inference products[object Object]:

  • The following input types are supported in non-quantization scenarios:

    • [object Object]: FLOAT16; [object Object]: FLOAT16; [object Object]: FLOAT16; [object Object]: null; [object Object]: null; [object Object]: null; [object Object]: null; [object Object]: FLOAT16
    • [object Object]: BFLOAT16; [object Object]: BFLOAT16; [object Object]: FLOAT32; [object Object]: null; [object Object]: null; [object Object]: null; [object Object]: null; [object Object]: BFLOAT16
    • [object Object]: FLOAT32; [object Object]: FLOAT32; [object Object]: FLOAT32; [object Object]: null; [object Object]: null; [object Object]: null; [object Object]: null; [object Object]: FLOAT32 (supported only when [object Object], [object Object], and [object Object] are all single-tensor)

  • The following input types are supported in quantization scenarios:

    • [object Object]: INT8; [object Object]: INT8; [object Object]: INT32; [object Object]: UINT64; [object Object]: null; [object Object]: null; [object Object]: null; [object Object]: INT8

  • The following input types are supported in fake-quantization scenarios:

    • [object Object]: FLOAT16; [object Object]: INT8; [object Object]: FLOAT16; [object Object]: null; [object Object]: null; [object Object]: FLOAT16; [object Object]: FLOAT16; [object Object]: FLOAT16
    • [object Object]: BFLOAT16; [object Object]: INT8; [object Object]: FLOAT32; [object Object]: null; [object Object]: null; [object Object]: BFLOAT16; [object Object]: BFLOAT16; [object Object]: BFLOAT16

  • Supported scenarios for different [object Object] values:

    • In A16W8 and A16W4 scenarios, [object Object] can be -1 or 0.

    • In A8W8, A8W4, and A4W4 scenarios, [object Object] can only be single -tensor when [object Object] is [object Object].

    • [object Object], [object Object], and [object Object] are of type aclTensorList, which is an array of aclTensor objects. In the following table, "S" indicates an aclTensorList consisting of one aclTensor, and "M" indicates an aclTensorList consisting of multiple aclTensors. For example, "SMS" indicates single-tensor [object Object], multi-tensor [object Object], and single-tensor [object Object].

      [object Object]undefined

  • The size of the last dimension for each tensor in [object Object] and [object Object] should be less than 65536. The last dimension of $x_i$ refers to the K-axis when [object Object] is not transposed or the M-axis when [object Object] is transposed. The last dimension of $weight_i$ refers to the N-axis when [object Object] is not transposed or the K-axis when [object Object] is transposed.

• Ascend 950PR/Ascend 950DT:

  [object Object]
  • The following data types are supported in non-quantization scenarios:

    • The following input parameters are null: scaleOptional, offsetOptional, antiquantScaleOptional, and antiquantOffsetOptional.

    • The data type combinations supported by the parameters that are not null must meet the requirements in the following table.

      [object Object]undefined
  [object Object][object Object]
  • The fake-quantization scenario supports the following data types:

    • The following input parameters are empty: scaleOptional and offsetOptional.

    • The combinations of data types supported by non-empty parameters must meet the requirements listed in the following table.

      [object Object]undefined

    • The combinations of antiquantScaleOptional, non-empty biasOptional, and antiquantOffsetOptional must meet the requirements listed in the following table. Here, g indicates the number of matmul groups.

      [object Object]undefined

    • Only the SSS and MMM scenarios are supported.

  [object Object][object Object]

  • The following input parameters are empty: offsetOptional, antiquantScaleOptional, and antiquantOffsetOptional.

  • Only the single-single-single scenario is supported.

  • The combinations of data types and dimensions supported by non-empty parameters must meet the requirements listed in the following table. Here, g indicates the number of matmul groups.

    [object Object]undefined
  [object Object][object Object]
  • Supported scenarios for different [object Object] values:

    • "S" stands for single-tensor, and "M" stands for multi-tensor, expressed in the sequence of [object Object], [object Object], [object Object]. For example, "SMS" indicates single-tensor [object Object], multi-tensor [object Object], and single-tensor [object Object].

      [object Object]undefined
  [object Object]

[object Object]

The following example is for reference only. For details, see .