RmsNormBackwardOperation
Description
Performs backward rmsnorm computation.
Definition
struct RmsNormBackwardParam {
uint8_t rsv[8] = {0};
};
Parameters
Member |
Type |
Default Value |
Description |
|---|---|---|---|
rsv[8] |
uint8_t |
{0} |
Reserved |
Input
Parameter |
Dimension |
Data Type |
Format |
Description |
|---|---|---|---|---|
dy |
[-1,…,-1] |
float16/float/bf16 |
ND |
Input gradient. The dimension is the same as that of x. The data type is the same as that of x. |
x |
[-1,…,-1] |
float16/float/bf16 |
ND |
Forward propagation input. |
rstd |
[-1,…,-1] |
float |
ND |
Intermediate result of forward propagation. |
gamma |
[-1,…,-1] |
float16/float/bf16 |
ND |
The data type is the same as that of x. The number of dimensions must be greater than 0 and less than the number of dimensions of x. The gamma dimension starts from the end must be the same as x. |
Output
Parameter |
Dimension |
Data Type |
Format |
Description |
|---|---|---|---|---|
dx |
[-1,…,-1] |
float16/float/bf16 |
ND |
Gradient of the forward input x. The dimension is the same as that of x. The data type is the same as that of x. |
dgamma |
[-1,…,-1] |
float |
ND |
Gradient of the forward input gamma. The dimension is the same as that of gamma. |
Restrictions
Currently, only the