GeluFusionPass

Description

Fuses small operators, such as Pow, Mul, Add, and Tanh, into the Gelu operator.

Formula:

• Scenario 1

  Before: After:

• Scenario 2

  Before: After:

  • Scenario 3

    Before: After:

  • Scenario 4

    Before: After:

    • Scenario 5

      Before: After:

    • Scenario 6

      Before: After:

      • Scenario 7

        Before: After:

      • Scenario 8

        Before: After:

Restrictions

• The dynamic shape scenario is not supported.
• Data type restrictions:
  • Atlas A2 training product/Atlas A2 inference product: The data type can be FLOAT32, FLOAT16, or BFLOAT16.

Availability

Atlas A2 training product/Atlas A2 inference product