[object Object]

Note: This API will be deprecated in later versions. Use the latest aclnnGroupedMatmulV5 API.

[object Object][object Object]undefined[object Object]

• Description: Implements grouped matrix multiplication, supporting non-uniform matrix dimension sizes across multiple groups. The basic function is matrix multiplication, for example, $y_i[m_i,n_i]=x_i[m_i,k_i] \times weight_i[k_i,n_i], i=1...g$, where $g$ indicates the number of groups and $m_i$, $k_i$, and $n_i$ define the shapes for each group. The following four scenarios are supported based on the tensor count of $x$, $weight$, and $y$:

  • Multi-tensor $x$, $weight$, and $y$. That is, the tensors of each group are independent.

  • Single-tensor $x$, multi-tensor $weight$ and $y$. In this case, use the optional parameter [object Object] to define the row-wise grouping of $x$. For example, [object Object] indicates that the first 10 rows of $x$ participate in the multiplication of the first group of matrices.

  • Multi-tensor $x$ and $weight$, single-tensor $y$. In this case, products of each matrix group multiplication are stored contiguously within a single tensor.

  • Single-tensor $x$ and $y$, multi-tensor $weight$. This is a hybrid configuration combining the preceding two cases.

    Note: "Single-tensor" means that tensors of all groups in a tensor list are concatenated into one tensor along the M-axis.

• Formula:

  • Non-quantization scenario:

    $$y_i=x_i\times weight_i + bias_i$$

  • Quantization scenario:

    $$y_i=(x_i\times weight_i + bias_i) * scale_i + offset_i$$

  • Dequantization scenario:

    $$y_i=(x_i\times weight_i + bias_i) * scale_i$$

  • Fake-quantization scenario:

    $$y_i=x_i\times (weight_i + antiquant\_offset_i) * antiquant\_scale_i + bias_i$$

[object Object]

Each operator has calls. First, [object Object] is called to obtain the input parameters and compute the required workspace size based on the process. Then, [object Object] is called to perform computation.

[object Object]

• Parameters

  [object Object]
  • [object Object]Atlas A2 training products/Atlas A2 inference products[object Object]:
    • x and weight support FLOAT16, BFLOAT16, and INT8.
    • y supports FLOAT16, BFLOAT16, INT8, and FLOAT32.
  • Ascend 950PR/Ascend 950DT:
    • x supports FLOAT16, BFLOAT16, and FLOAT32.
    • weight supports FLOAT16, BFLOAT16, FLOAT32, and INT8.
    • y supports FLOAT16, BFLOAT16, and FLOAT32.
    • scaleOptional and offsetOptional are not supported.

• Return

  [object Object]: status code. For details, see .

  The first-phase API implements input parameter validation. The following errors may be thrown.

  [object Object]

[object Object]

• Parameters

  [object Object]

• Return

  [object Object]: status code. For details, see .

[object Object]

• Deterministic computation:

  • [object Object] defaults to a deterministic implementation.

• [object Object]Atlas A2 training products/Atlas A2 inference products[object Object]:

  • The following input types are supported in non-quantization scenarios:

    • [object Object]: FLOAT16; [object Object]: FLOAT16; [object Object]: FLOAT16; [object Object]: null; [object Object]: null; [object Object]: null; [object Object]: null; [object Object]: FLOAT16
    • [object Object]: BFLOAT16; [object Object]: BFLOAT16; [object Object]: FLOAT32; [object Object]: null; [object Object]: null; [object Object]: null; [object Object]: null; [object Object]: BFLOAT16

  • The following input type is supported in quantization scenarios:

    • [object Object]: INT8; [object Object]: INT8; [object Object]: INT32; [object Object]: UINT64; [object Object]: null; [object Object]: null; [object Object]: null; [object Object]: INT8

  • The following input types are supported in fake-quantization scenarios:

    • [object Object]: FLOAT16; [object Object]: INT8; [object Object]: FLOAT16; [object Object]: null; [object Object]: null; [object Object]: FLOAT16; [object Object]: FLOAT16; [object Object]: FLOAT16
    • [object Object]: BFLOAT16; [object Object]: INT8; [object Object]: FLOAT32; [object Object]: null; [object Object]: null; [object Object]: BFLOAT16; [object Object]: BFLOAT16; [object Object]: BFLOAT16

  • If [object Object] is passed, it must be a non-negative ascending array, and its length cannot be 1.

  • The following scenarios are supported: "S" stands for single-tensor, and "M" stands for multi-tensor, expressed in the sequence of x, weight, y. For example, "SMS" indicates single-tensor x, multi-tensor weight, and single-tensor y.

    [object Object]undefined

  • The size of the last dimension for each tensor in [object Object] and [object Object] should be less than 65536. The last dimension of $x_i$ refers to the K-axis when [object Object] is false or the M-axis when [object Object] is true. The last dimension of $weight_i$ refers to the N-axis when [object Object] is false or the K-axis when [object Object] is true.

  • The size of each dimension for every tensor in [object Object] and [object Object], after 32-byte alignment, should be less than the maximum value of INT32 (2147483647).

• Ascend 950PR/Ascend 950DT:

  [object Object]
  • The following data types are supported in non-quantization scenarios:

    • If [object Object] is passed, it must be a non-negative ascending array, and its length cannot be 1.

    • The following input parameters are empty: scaleOptional, offsetOptional, antiquantScaleOptional, and antiquantOffsetOptional.

    • The data type combinations supported by the parameters that are not empty must meet the requirements in the following table:

      [object Object]undefined
  [object Object][object Object]

  • The fake-quantization scenario supports the following data types:

    • The following input parameters are empty: scaleOptional and offsetOptional.

    • The combinations of data types supported by non-empty parameters must meet the requirements listed in the following table.

      [object Object]undefined

    • The antiquantScaleOptional, non-empty biasOptional, and antiquantOffsetOptional must meet the requirements listed in the following table.

      [object Object]undefined

  • Only the MMM scenario is supported.

    [object Object]
  [object Object]
  [object Object]
  [object Object]

[object Object]

The following example is for reference only. For details, see .