Constraints
- Except the scale, offset, and rstd tensors, the last dimensions of all input and output tensors must be the same and 32-byte aligned.
- rstd is supported only by the
Atlas A2 training products /Atlas A2 inference products andAtlas A3 inference products /Atlas A3 training products and cannot be set together with precisionMode and modelType. It cannot be used in quantization scenarios. - precisionMode cannot be set together with rstd and modelType, and cannot be used in quantization scenarios.
- modelType cannot be set together with rstd and precisionMode, and cannot be used in quantization scenarios.
- Currently, dynamicQuantType supports only DYNAMIC_QUANT_SYMMETRIC. During dynamic symmetric quantization, the size of the last input dimension must be less than or equal to 12288.
- Currently, the
Atlas training products supports only non-quantization and non-fusion rmsnorm. - RMS_NORM_PRENORM and RMS_NORM_POSTNORM are supported only by
Atlas A2 training products /Atlas A2 inference products andAtlas A3 inference products /Atlas A3 training products . - On the
Atlas 200I/500 A2 inference products , only the float16 data type is supported, and layerType is RMS_NORM_NORM, quantType is QUANT_UNQUANT or QUANT_INT8, and dynamicQuantType is 0, that is, dynamicQuant is not supported. - Atlas 350 accelerator card: layerType only supports configuration as RMS_NORM_NORM.
Parent topic: RmsNormOperation