Parameters

Member	Description
RmsNormParam	RMSNORM parameter. RMS_NORM_UNDEFINED (default): undefined. This default type is unavailable. You must set this parameter. RMS_NORM_NORM: NORM parameter. RMS_NORM_PRENORM: PRENORM parameter RMS_NORM_POSTNORM: POSTNORM parameter.
PrecisionMode	PRECISIONMODE parameter. HIGH_PRECISION_MODE (default): The float type is used for intermediate calculation. HIGH_PERFORMANCE_MODE: The float16 type is used for intermediate calculation.
ModelType	MODELTYPE parameter. LLAMA_MODEL (default): The formula of Llama rmsnorm is used. GEMMA_MODEL: The formula of Gemma rmsnorm is used.
layerType	Normalization type. The options are as follows: RMS_NORM_UNDEFINED (default): undefined. RMS_NORM_NORM: NORM parameter. RMS_NORM_PRENORM: PRENORM parameter RMS_NORM_POSTNORM: POSTNORM parameter.
normParam	NORM parameter. For details, see Table 1.
preNormParam	PRENORM parameter. For details, see Table 2.
postNormParam	POSTNORM parameter. For details, see Table 3.
rsv[8]	Reserved

**Table 1** NormParam members
Parameter	Type	Default Value	Description
quantType	QuantType	QUANT_UNQUANT	Quantization type. Currently, the following types are supported: QUANT_UNQUANT QUANT_INT8
epsilon	float	1e-5	Epsilon, which is added to the denominator during normalization to prevent division by 0.
layerNormEps	float	1e-5	Epsilon, which is not used currently.
rstd	bool	false	If this parameter is set to true, the trained rmsnormforward operator is used. This operator is supported only by the Atlas A2 training products/Atlas A2 inference products and Atlas A3 inference products/Atlas A3 training products. This parameter cannot be set together with precisionMode and modelType. rstd is not supported in quantization scenarios.
precisionMode	PrecisionMode	HIGH_PRECISION_MODE	Values: HIGH_PRECISION_MODE (default): The float type is used for intermediate calculation. HIGH_PERFORMANCE_MODE: The float16 type is used for intermediate calculation. This parameter cannot be set together with rstd and modelType. The input type can only be float16. In quantization scenarios, precisionMode is not supported. If this parameter is configured in this scenario, the error message "ERROR_INVALID_PARAM" is returned.
modelType	ModelType	LLAMA_MODEL	When the value is set to GEMMA_MODEL, the rmsnorm calculation formula of the gemma model is used. Values: LLAMA_MODEL (default ): rms norm calculation formula of Llama. GEMMA_MODEL: rms norm calculation formula of GEMMA. This parameter cannot be used together with rstd and precisionMode. In quantization scenarios, modelType is not supported. If this parameter is configured in this scenario, the error message "ERROR_INVALID_PARAM" is returned.
dynamicQuantType	DynamicQuantType	DYNAMIC_QUANT_UNDEFINED	Dynamic quantization type. The default value is DYNAMIC_QUANT_UNDEFINED. The current version does not support asymmetric dynamic quantization.
rsv[32]	uint8_t	{0}	Reserved

In 8.0.RC2 and later versions, normParam does not support the quantInputScale and quantInputOffset quantization parameters.

Atlas 350 accelerator card supports only QUANT_UNQUANT. epsilon and rstd can only be set to false. Other parameters do not take effect.

**Table 2** PreNormParam members
Parameter	Type	Default Value	Description
quantType	QuantType	QUANT_UNQUANT	Quantization type. Currently, the following types are supported: QUANT_UNQUANT QUANT_INT8
epsilon	float	1e-5	Epsilon, which is added to the denominator during normalization to prevent division by 0.
hasBias	bool	false	Whether to add a bias. Set this parameter to true when beta needs to be input. In a quantization scenario, hasBias is not supported. If this parameter is set in this scenario, the error message "ERROR_INVALID_PARAM" is returned.
rsv[23]	uint8_t	{0}	Reserved

In 8.0.RC2 and later versions, PreNormParam does not support the quantInputScale and quantInputOffset quantization parameters.

Atlas 350 accelerator card supports only QUANT_UNQUANT. epsilon and hasBias can only be set to false.

**Table 3** PostNormParam members
Parameter	Type	Default Value	Description
quantType	QuantType	QUANT_UNQUANT	Quantization type. Currently, the following types are supported: QUANT_UNQUANT QUANT_INT8
epsilon	float	1e-5	Epsilon, which is added to the denominator during normalization to prevent division by 0.
hasBias	bool	false	Whether to add a bias. Set this parameter to true when beta needs to be input. In a quantization scenario, hasBias is not supported. If this parameter is set in this scenario, the error message "ERROR_INVALID_PARAM" is returned.
rsv[23]	uint8_t	{0}	Reserved

Parent topic: RmsNormOperation