Normal Matmul and Bias

Applicable Products

Description

Matmul matrix multiplication. The hasBias parameter can be used to control whether to add a bias.

Formula

Multiplies tensors A (x) and B (weight) to output tensor C.

Adds a bias.

Parameter Configuration

Input

Output

Description

The following table lists the combinations of input and output attributes. The combinations that are not listed in the table are not supported.

Figure 1 Combinations of input and output attributes

• Atlas inference products : Combinations 7 to 13 are not supported.
• If transposeA is set to true, combinations 2, 5, 8, and 11 are not supported.
• The data types of all input and output tensors are the same.
• When weight has four dimensions, the values of k and n are integer multiples of 16.
• Combination 13 ensures the performance for only the shape: m = 1 to 256, k = 7168, and n = 256. For other shapes, the performance may deteriorate.

OP Usage and Typical Scenarios

For details about how to use OP, see the usage process in Operator Usage Guide (ATB C++ APIs). For details about how to construct the Operation parameter as instructed in Single-operator, see the parameter construction in the following scenarios.

• Scenario 1

  // Parameter construction
  atb::infer::LinearParam param;
  param.transposeA = false;
  param.transposeB = false;
  param.hasBias = false;
  param.outDataType = ACL_DT_UNDEFINED;
  param.enAccum = false;
  param.matmulType = MATMUL_UNDEFINED;
  param.quantMode = QUANT_UNDEFINED;

  # Example
  >>> x
  tensor([[1, 2],
          [3, 4]])
  >>> weight
  tensor([[1, 2, 3],
          [4, 5, 6]])
  >>> output
  tensor([[9, 12, 15],
          [19, 26, 33]])
  # 9  = 1 * 1 + 2 * 4
  # 12 = 1 * 2 + 2 * 5
  # 15 = 1 * 3 + 2 * 6
  # 19 = 3 * 1 + 4 * 4
  # 26 = 3 * 2 + 4 * 5
  # 33 = 3 * 3 + 4 * 6

• Scenario 2

  // Parameter construction
  atb::infer::LinearParam param;
  param.transposeA = true;
  param.transposeB = true;
  param.hasBias = false;
  param.outDataType = ACL_DT_UNDEFINED;
  param.enAccum = false;
  param.matmulType = MATMUL_UNDEFINED;
  param.quantMode = QUANT_UNDEFINED;

  # Based on the transposition, the computation in this example is the same as that in the previous example.
  >>> x
  tensor([[1, 3],
          [2, 4]])
  >>> weight
  tensor([[1, 4],
          [2, 5],
          [3, 6]])
  >>> output
  tensor([[9, 12, 15],
          [19, 26, 33]])

• Scenario 3

  // Parameter construction
  atb::infer::LinearParam param;
  param.transposeA = false;
  param.transposeB = false;
  param.hasBias = true;
  param.outDataType = ACL_DT_UNDEFINED;
  param.enAccum = false;
  param.matmulType = MATMUL_UNDEFINED;
  param.quantMode = QUANT_UNDEFINED;

  # Example
  >>> x
  tensor([[1, 2],
          [3, 4]])
  >>> weight
  tensor([[1, 2, 3],
          [4, 5, 6]])
  >>> bias
  tensor([1, 2, 3])
  >>> output
  tensor([[10, 14, 18],
          [20, 28, 36]])
  # 10 = 1 * 1 + 2 * 4 + 1
  # 14 = 1 * 2 + 2 * 5 + 2
  # 18 = 1 * 3 + 2 * 6 + 3
  # 20 = 3 * 1 + 4 * 4 + 1
  # 28 = 3 * 2 + 4 * 5 + 2
  # 36 = 3 * 3 + 4 * 6 + 3

Function Constraints

• Some scenarios are not supported when transposeA is set to true.
• outDataType = ACL_DT_UNDEFINED.
• enAccum = false.