GroupNorm

Applicability

Product	Supported
Atlas 350 Accelerator Card	√
Atlas A3 training product/Atlas A3 inference product	√
Atlas A2 training product/Atlas A2 inference product	√
Atlas 200I/500 A2 inference product	x
Atlas inference product AI Core	x
Atlas inference product Vector Core	x
Atlas training product	x

Function Usage

A general formula for normalizing a feature is as follows:

$\text{[math]}$

In this formula, i indicates the index of the feature. $\text{[math]}$ and $\text{[math]}$ indicate the values of the feature before and after normalization, respectively. μ and σ indicate the mean value and standard deviation of the feature, and their formulas are as follows:

$\text{[math]}$

ε is a small constant. S indicates the set of data involved in the calculation, and m indicates the set size. The main difference between different types of feature standardization methods (such as BatchNorm, LayerNorm, InstanceNorm, and GroupNorm) lies in the selection of data sets involved in calculation. The following describes how to select data sets for different Norm operators for computation.

For an input with the shape of [N, C, H, W], GroupNorm divides each [C, H, W] into groupNum groups in the C dimension and then normalizes each group. Finally, the standardized feature is scaled and translated. The scaling parameter γ and the translation parameter β are trainable.

$\text{[math]}$

Prototype

Allocate the temporary space through the API framework.

template <typename T, bool isReuseSource = false>
__aicore__ inline void GroupNorm(const LocalTensor<T>& output, const LocalTensor<T>& outputMean, const LocalTensor<T>& outputVariance, const LocalTensor<T>& inputX, const LocalTensor<T>& gamma, const LocalTensor<T>& beta, const T epsilon, GroupNormTiling& tiling)

Pass the temporary space through the sharedTmpBuffer input parameter.

template <typename T, bool isReuseSource = false>
__aicore__ inline void GroupNorm(const LocalTensor<T>& output, const LocalTensor<T>& outputMean, const LocalTensor<T>& outputVariance, const LocalTensor<T>& inputX, const LocalTensor<T>& gamma, const LocalTensor<T>& beta, const LocalTensor<uint8_t>& sharedTmpBuffer, const T epsilon, GroupNormTiling& tiling)

Parameters

**Table 1** Template parameters
Parameter	Description
T	Data type of the operand. For the Atlas 350 Accelerator Card, the supported data types are half and float. For the Atlas A3 training product/Atlas A3 inference product, the supported data types are half and float. For the Atlas A2 training product/Atlas A2 inference product, the supported data types are half and float.
isReuseSource	Whether the source operand can be modified. The default value is false. If developers allow the source operand to be modified, enable this parameter, to reduce memory space usage. If this parameter is set to true, the inputX memory space is reused during internal computation of this API to reduce memory space usage. If this parameter is set to false, the inputX memory space is not reused during internal computation of this API. This parameter can be enabled for float input data but cannot be enabled for half input data. For details about how to use isReuseSource, see Example 4.

**Table 2** API parameters
Parameter	Input/Output	Description
output	Output	Destination operand, which is the result of scaling and translation calculation on the standardized input. The shape is [N, C, H, W]. The type is LocalTensor, and TPosition can be VECIN, VECCALC, or VECOUT.
outputMean	Output	Destination operand, which indicates the mean value. The shape is [N, groupNum]. The type is LocalTensor, and TPosition can be VECIN, VECCALC, or VECOUT.
outputVariance	Output	Destination operand, which indicates the variance. The shape is [N, groupNum]. The type is LocalTensor, and TPosition can be VECIN, VECCALC, or VECOUT.
inputX	Input	Source operand. The shape is [N, C, H, W]. The type is LocalTensor, and TPosition can be VECIN, VECCALC, or VECOUT.
gamma	Input	Source operand, which indicates the scaling parameter. The value range of this parameter is [–100, 100]. The shape is [C]. The type is LocalTensor, and TPosition can be VECIN, VECCALC, or VECOUT.
beta	Input	Source operand, which indicates the translation parameter. The value range of this parameter is [–100, 100]. The shape is [C]. The type is LocalTensor, and TPosition can be VECIN, VECCALC, or VECOUT.
sharedTmpBuffer	Input	This parameter is used to store intermediate variables during complex internal API computation and is provided by developers. The type is LocalTensor, and TPosition can be VECIN, VECCALC, or VECOUT. For details about how to obtain the temporary space size (BufferSize), see GroupNorm Tiling.
epsilon	Input	Weight coefficient for preventing division by zero. The data type must be the same as that of inputX or output.
tiling	Input	Tiling information of input data. For details about how to obtain the tiling information, see GroupNorm Tiling.

Returns

None

Constraints

For details about the operand address alignment requirements, see General Address Alignment Restrictions.
Currently, only the ND format is supported.

Examples

For a complete call example, see GroupNorm operator sample.

// output: tensor for storing the GroupNorm computation result
// outputMean: output mean value of each group
// outputVariance: output variance of each group
// inputX: input data X, with shape [N, C, H, W]
// gamma: scaling parameter γ of LayerNorm, with shape [C]
// beta: bias parameter β of LayerNorm, with shape [C]
// epsilon: coefficient ε for preventing division by zero
// tiling: pre-computed tiling information, including parameters such as the number of groups and dimensions

// Use the GroupNorm API to implement Group Normalization.
// If the data type T is float and inputX can be modified, you can set isReuseSource to true to reuse the inputX memory space to save memory.
AscendC::GroupNorm<T, isReuseSource>(
    output,           // Output: normalized, scaled, translated result
    outputMean,       // Output: mean value of each group
    outputVariance,   // Output: variance of each group
    inputX,           // Input: original feature map
    gamma,            // Input: scaling parameter γ
    beta,             // Input: bias parameter β
    epsilon,          // Input: coefficient ε for preventing division by zero
    tiling            // Input: tiling scheduling information
);

The following is an example:

Input (inputXLocal, shape: [2, 8, 4, 2]):
[  0  1  2  3  4  5  6  7  8  9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 
  32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63
  64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 
  96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 ]
Input (gammaLocal, shape: [8]):
[ 0 1 2 3 4 5 6 7 ]
Input (betaLocal, shape: [8]):
[ 0 1 2 3 4 5 6 7 ]
Output (dstLocal):
[ 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 
  1.1084652 1.3253956 1.542326 1.7592564 1.9761869 2.1931171 2.4100475 2.6269782 
  -1.2539563 -0.8200953 -0.38623452 0.047626257 0.48148715 0.91534793 1.3492088 1.7830696 
  3.3253956 3.9761868 4.626978 5.277769 5.9285607 6.579352 7.230143 7.8809347 
  -2.5079126 -1.6401906 -0.77246904 0.095252514 0.9629743 1.8306959 2.6984177 3.5661392 
  5.542326 6.626978 7.71163 8.796282 9.880934 10.965586 12.050238 13.134891 
  -3.7618694 -2.4602861 -1.1587038 0.14287853 1.4444613 2.7460437 4.0476265 5.349209 
  7.7592564 9.277769 10.796282 12.314795 13.833308 15.351821 16.870335 18.388847 
  0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 
  1.1084652 1.3253956 1.542326 1.7592564 1.9761869 2.1931171 2.4100475 2.6269782 
  -1.2539563 -0.8200953 -0.38623452 0.047626257 0.48148715 0.91534793 1.3492088 1.7830696 
  3.3253956 3.9761868 4.626978 5.277769 5.9285607 6.579352 7.230143 7.8809347 
  -2.5079126 -1.6401906 -0.77246904 0.095252514 0.9629743 1.8306959 2.6984177 3.5661392 
  5.542326 6.626978 7.71163 8.796282 9.880934 10.965586 12.050238 13.134891 
  -3.7618694 -2.4602861 -1.1587038 0.14287853 1.4444613 2.7460437 4.0476265 5.349209 
  7.7592564 9.277769 10.796282 12.314795 13.833308 15.351821 16.870335 18.388847 ]
Output (meanLocal):
[ 7.5 23.5 39.5 55.5 71.5 87.5 103.5 119.5 ]
Output (varianceLocal):
[ 21.25 21.25 21.25 21.25 21.25 21.25 21.25 21.25 ]

Parent topic: Normalization