GeluFusionPass

Description

Fuses small operators, such as Pow, Mul, Add, and Tanh, into the Gelu operator.

The formula is as follows:

$\text{[math]}$

Scenario 1
Before: After:
Scenario 2
Before: After:
- Scenario 3
  Before: After:
- Scenario 4
  Before: After:
  - Scenario 5
    Before: After:
  - Scenario 6
    Before: After:
    - Scenario 7
      Before: After:
    - Scenario 8
      Before: After:

Constraints

Dynamic shapes are not supported.
Data type restrictions:
- Atlas A2 training products/Atlas A2 inference products: The data type can be FLOAT32, FLOAT16, or BFLOAT16.

Applicable Products

Atlas A2 training products/Atlas A2 inference products

Parent topic: Graph Fusion Patterns