Parameters

Member	Type	Default Value	Value Range	Description
transposeA	bool	false	false/true	Whether to transpose matrix A. If the value is true, transpose is performed. Otherwise, transpose is not performed.
transposeB	bool	true	false/true	Whether to transpose matrix B. If the value is true, transpose is performed. Otherwise, transpose is not performed.
hasBias	bool	true	false/true	Whether to add a bias. If the value is true, the bias is added. Otherwise, the bias is not added.
outDataType	aclDataType	ACL_DT_UNDEFINED	ACL_DT_UNDEFINED/ACL_FLOAT16/ACL_BF16	Data type of the output tensor. The default value indicates that the output tensor data type is automatically derived from the input tensors.
enAccum	bool	false	false/true	Whether to enable accumulation, that is, the matmul+inplace add fusion scenario.
matmulType	MatmulType	MATMUL_UNDEFINED	MATMUL_UNDEFINED/MATMUL_EIN_SUM	Type of the matrix multiplication function. MATMUL_UNDEFINED: includes other features except Einstein multiplication. MATMUL_EIN_SUM: Einstein multiplication is enabled.
quantMode	QuantMode	QUANT_UNDEFINED	QUANT_UNDEFINED PER_CHANNEL PER_TOKEN	Linear quantization type. QUANT_UNDEFINED: default value in non-quantization scenarios. The function is the same as that of PER_CHANNEL in quantization scenarios. PER_CHANNEL: per_channel quantization. PER_TOKEN: per_token quantization.
rsv[21]	uint8_t	{0}	[0]	Reserved

Detailed Description

transposeA and transposeB

Based on the configurations of transposeA and transposeB, the shapes of the transposed matrices A and B must meet the dimension relationship for matrix multiplication.
The values of transposeA and transposeB are restricted in some functions.
transposeA and transposeB can improve operator performance in special scenarios. The benefits vary depending on the shape. Using normal matrix multiplication as an example and assuming that shapes of the matrix A and the matrix B are respectively [m, k] and [k, n]:
- When m is an integer multiple of 256 and k is not an integer multiple of 256, set transposeA to true to transpose matrix A to improve performance. In this case, the shape of matrix A is [k, m].
- When k is an integer multiple of 256 and n is not an integer multiple of 256, set transposeB to true to transpose matrix B to improve performance. In this case, the shape of matrix B is [n, k].

In the following function description, transposeA and transposeB are set to false by default. The following table lists the dimension changes.

Original Dimension of Matrix A	Matrix A Dimension When transposeA Is True
[m, k]	[k, m]
[batch, m, k]	[batch, k, m]

Original Dimension of Matrix B	Matrix B Dimension When transposeB Is True
[k, n]	[n, k]
[batch, k, n]	[batch, n, k]
[1, n / 16, k, 16]	[1, k / 16, n, 16]
[batch, n / 16, k, 16]	[batch, k / 16, n, 16]
[1, n / 32, k, 32]	[1, k / 32, n, 32]
[batch, n / 32, k, 32]	[batch, k / 32, n, 32]

hasBias

When enAccum is set to true, hasBias can only be set to false.
The value of hasBias is restricted in some functions.

outDataType

In the dequantization scenario, set to the data type of the output tensor. Currently, ACL_FLOAT16 and ACL_BF16 are supported. In other scenarios, set to the default value ACL_DT_UNDEFINED.

enAccum

When hasBias is set to true, enAccum can only be set to false.
If this parameter is set to true, the matmul + inplaceAdd fusion function is triggered. For details about the design and restrictions of this function, see Fusion of Matmul and InplaceAdd.

matmulType

In Einstein multiplication scenarios, set this parameter to MATMUL_EIN_SUM. In other scenarios, set this parameter to MATMUL_UNDEFINED.
When matmulType is set to MATMUL_EIN_SUM:
- transposeA can only be set to false.
- hasBias can only be set to false.
- enAccum can only be set to false.
- outDataType can only be set to the default value ACL_DT_UNDEFINED.

quantMode

When outDataType is set to ACL_DT_UNDEFINED, the scenario is a non-quantization scenario, and only the input value QUANT_UNDEFINED is supported.
When outDataType is set to ACL_FLOAT16 or ACL_BF16, the scenario is a quantization scenario:
- If the input value is QUANT_UNDEFINED or PER_CHANNEL, per_channel quantization is used. When hasBias is set to true, asymmetric quantization is used. When hasBias is set to false, symmetric quantization is used.
- If the input value is PER_TOKEN, per_token quantization is used. In this case, only symmetric quantization with hasBias set to false is supported.

Parent topic: LinearOperation