AddLayerNormDynamicQuantFusionPass

Description

Fuses AddLayerNorm and one or two downstream DynamicQuant operators into the AddLayerNormQuant operator.

There may be one or two structures in the dashed-line box.

When the optional smooth input of DynamicQuant exists, it will be placed in the position corresponding to the scale input of AddLayerNormQuant.

The smooth operation of DynamicQuant can also be expressed by Mul. In this scenario, the optional smooth input of DynamicQuant must be empty.

Constraints

• For dtype:
  • For Ascend 950PR/Ascend 950DT, the data type of AddLayerNorm x1 can be fp16, bf16, or fp32.
  • For Atlas A2 training products/Atlas A2 inference products, the data type of AddLayerNorm x1 can be fp16 or bf16.
  • The data types of inputs x2, gamma, beta, and bias of AddLayerNorm must be the same as that of x1.
  • The input x of DynamicQuant must be of the same data type as x1 of AddLayerNorm.
  • For Ascend 950PR/Ascend 950DT, if DynamicQuant has the smooth parameter, the data type of smooth must be the same as that of x1 of AddLayerNorm.

• For the graph structure:
  • AddLayerNorm can have only one or two fan-outs for its output 0, connecting to a Mul node or a DynamicQuant node.
  • When AddLayerNorm has two output0, the two downstream nodes must be Mul or DynamicQuant at the same time.
  • When the downstream node of output0 of AddLayerNorm is Mul, the downstream node of Mul can have only one DynamicQuant node, and DynamicQuant cannot have the optional input smooth.

Applicable Products

Atlas A2 training products/Atlas A2 inference products

Ascend 950PR/Ascend 950DT