Formula

$\text{[math]}$

beta is used in the rmsnorm quantization scenario.

$\text{[math]}$

gamma is a trainable parameter, and epsilon is a floating-point value whose value range is (0, 1], and is used to avoid division by zero.

$\text{[math]}$

Parent topic: RmsNormOperation