CohereLayerNormOperation

Hardware Model	Supported or Not	Remarks
Atlas 350 accelerator card	x	-
Atlas A3 inference products/Atlas A3 training products	√	float16 and bf16 data types allowed.
Atlas A2 training products/Atlas A2 inference products	√	float16 and bf16 data types allowed.
Atlas training products	x	-
Atlas inference products	x	-
Atlas 200I/500 A2 inference products	x	-

Normalizes the input data at the network layer to the [0, 1] range based on the -1 dimension.

struct CohereLayerNormParam {
    float epsilon = 1e-5;
    uint8_t rsv[32] = {0};
};

Member	Type	Default Value	Value Range	Mandatory or Not	Description
epsilon	float	1e-5	Greater than 0	No	Epsilon, which is added to the denominator during normalization to prevent division by 0.
rsv	uint8_t	{0}	-	No	Reserved parameter.

Function
For the Command R Plus model, multi-batch data is used to indicate that the normalization operation is performed based on the last dimension.
Formula
$\text{[math]}$

E[, -1] and Var[, -1] are the mean and variance of the last dimension, respectively. $\text{[math]}$ is epsilon, and $\text{[math]}$ is the weight.

Input

Parameter	Dimension	Data Type	Format	Description
x	[dim_0, dim_1, ..., dim_n]	float16/bf16	ND	The size of the last dimension must be a multiple of 32. The dimension can only be 3 or 4.
gamma	[dim_n-1, dim_n]	float16/bf16	ND	The size of the last dimension must be a multiple of 32. The dimension can only be 2. The last two dimensions of x and gamma must be the same.

Output
Parameter

Dimension

Data Type

Format

Description

result

[dim_0, dim_1, ..., dim_n]

float16/bf16

ND

The data formats of x, result, and gamma are the same.

Parameter	Dimension	Data Type	Format	Description
result	[dim_0, dim_1, ..., dim_n]	float16/bf16	ND	The data formats of x, result, and gamma are the same.

Parent topic: atb/infer_op_params.h