Description

Performs matrix multiplication on the two matrices A and B, and adds an offset to, fuses InplaceAdd, or dequantizes the matrix multiplication result.

The operator receives the x and weight input tensors as matrices A and B for multiplication. The transposeA and transposeB parameters can be used to transpose matrices A and B before matrix multiplication. Matrices A and matrix B obtained after parameter transposition need to meet a dimension relationship for matrix multiplication. For example, when transposeA is false and transposeB is true, shapes of x and weight may be [m, k] and [n, k], respectively.

This operator supports floating-point and quantization scenarios. If the value of outDataType is ACL_DT_UNDEFINED, the floating-point scenario is used. Otherwise, the quantization scenario is used.

This operator can be used in two quantization scenarios. If the value of quantMode is QUANT_UNDEFINED or PER_CHANNEL, the per-channel quantization is used. If the value is PER_TOKEN, the per-token quantization is used.

This operator can be used in the Einstein multiplication scenario. If the value of matmulType is MATMUL_EIN_SUM, the Einstein multiplication is used.