RmsNormBackwardOperation
Applicable Products
Product |
Supported |
|---|---|
Atlas 350 accelerator cards |
x |
√ |
|
√ |
|
x |
|
x |
|
x |
Description
Performs backward RmsNorm computation.
Definition
1 2 3 | struct RmsNormBackwardParam { uint8_t rsv[8] = {0}; }; |
Parameters
Member |
Type |
Default Value |
Description |
|---|---|---|---|
rsv[8] |
uint8_t |
{0} |
Reserved |
Input
Parameter |
Dimension |
Data Type |
Format |
Description |
|---|---|---|---|---|
dy |
[dim_0, dim_1, ..., dim_n] |
float16/float/bf16 |
ND |
Input gradient. The data type and dimension are the same as those of x. |
x |
[dim_0, dim_1, ..., dim_n] |
float16/float/bf16 |
ND |
Forward propagation input. |
rstd |
[dim_0, dim_1, ..., dim_n] |
float |
ND |
Intermediate result of forward propagation. If the number of dimensions of the ith dimension is greater than or equal to x minus the number of gamma dimensions, the size is 1. Otherwise, the size is the same as the size of the corresponding dimension in x. |
gamma |
[dim_0, dim_1, ..., dim_n] |
float16/float/bf16 |
ND |
The data type is the same as that of x. The number of dimensions must be greater than 0 and less than the number of dimensions of x. The gamma dimension starts from the end must be the same as x. |
Output
Parameter |
Dimension |
Data Type |
Format |
Description |
|---|---|---|---|---|
dx |
[dim_0, dim_1, ..., dim_n] |
float16/float/bf16 |
ND |
Gradient of the forward input x. The data type and dimension are the same as those of x. |
dgamma |
[dim_0, dim_1, ..., dim_n] |
float |
ND |
Gradient of the forward input gamma. The dimension is the same as that of gamma. |