BatchMatMul2MulFusionPass

Description

MatMul/MatMulv2/MatMulV3/BatchMatmul/BatchMatMulV2/BatchMatMulV3 operators with k=1 exist in the network, and the performance is poor. This graph fusion converts MatMul/MatMulv2/MatMulV3/BatchMatmul/BatchMatMulV2/BatchMatMulV3 into mul to solve the performance problem.

In Atlas 350 Accelerator Card, only MatMul/MatMulv2/MatMulV3 nodes are fused, and BatchMatmul/BatchMatMulV2/BatchMatMulV3 nodes are not fused.

If the input adj is set to true, insert the reshape operator before the input.

Restrictions

• It applies only to static scenarios. The input does not contain bias.
• The dtype meets the following conditions:
  • Float32 for both input and output
  • Float16 or BFloat16 for both input and output

Availability

Atlas A2 training product/Atlas A2 inference product

Atlas A3 training product/Atlas A3 inference product

Atlas 350 Accelerator Card