LayerNormalization

Description

Performs layer normalization. Normalization is performed on a specified layer.

Input

• x: input tensor of type float16.
• scale: input tensor of type float16, specifying the scale factor.
• B (optional): input tensor of type float16, specifying the offset.

Attribute

• axis: int, indicating the first dimension to be normalized. The default value is -1.
• epsilon: float. The default value is 1e-05. A small value is added to var to avoid division by 0.
• stash_type: int. The default value is 1, specifying the type of mean and InvStdDev.

Output

• y: tensor after normalization. The data type is float16 or float.
• mean: average value, which is an optional output.
• InvStdDev: (optional) reciprocal of the standard deviation.

Constraints

None.

ONNX Opset Support

Opset v17/v18