LayerNormGradBeta

Applicability

Product	Supported
Atlas 350 Accelerator Card	√
Atlas A3 training product/Atlas A3 inference product	√
Atlas A2 training product/Atlas A2 inference product	√
Atlas 200I/500 A2 inference product	x
Atlas inference product AI Core	√
Atlas inference product Vector Core	x
Atlas training product	x

Function Usage

Obtains the reverse beta and gamma values and outputs pdx, gamma, and beta when used in conjunction with LayerNormGrad.

The formulas are as follows:

$\text{[math]}$

Prototype

Due to the complex computation involved in the internal implementation of this API, additional temporary space is required to store intermediate variables generated during computation. The method of obtaining the temporary space size (BufferSize) is as follows: Obtain the required maximum and minimum temporary space sizes using the GetLayerNormGradBetaMaxMinTmpSize API provided in LayerNormGradBeta Tiling. The minimum space can ensure correct functionality, while the maximum space is used to improve performance.

The temporary space can be allocated through the API framework or passed by developers through the sharedTmpBuffer input parameter. Therefore, there are two types of function prototypes for the LayerNormGradBeta API.

Pass the temporary space through the sharedTmpBuffer input parameter.

template <typename T, bool isReuseSource = false>
__aicore__ inline void LayerNormGradBeta(const LocalTensor<T>& outputPdGamma, const LocalTensor<T>& outputPdBeta, const LocalTensor<T>& resForGamma, const LocalTensor<T>& inputDy, const LocalTensor<uint8_t>& sharedTmpBuffer, const LayerNormGradBetaTiling& tiling)

This method enables developers to allocate and manage the temporary buffer space on their own, and reuse the buffer after calling the API, so that the buffer is not repeatedly allocated or deallocated, improving the flexibility and buffer utilization.

Allocate the temporary space through the API framework.

template <typename T, bool isReuseSource = false>
__aicore__ inline void LayerNormGradBeta(const LocalTensor<T>& outputPdGamma, const LocalTensor<T>& outputPdBeta, const LocalTensor<T>& resForGamma, const LocalTensor<T>& inputDy, LayerNormGradBetaTiling& tiling)

When using this method, developers do not need to allocate the space, but must reserve the required size for the space.

Parameters

**Table 1** Template parameters
Parameter	Description
T	Data type of the operand. For the Atlas 350 Accelerator Card, the supported data types are half and float. For the Atlas A3 training product/Atlas A3 inference product, the supported data types are half and float. For the Atlas A2 training product/Atlas A2 inference product, the supported data types are half and float. For the Atlas inference product AI Core, the supported data types are half and float.
isReuseSource	Whether the source operand can be modified. The default value is false. If developers allow the source operand to be modified, enable this parameter, to reduce memory space usage. If this parameter is set to true, the inputDy memory space is reused during internal computation of this API to save the memory space. If this parameter is set to false, the inputDy memory space is not reused. This parameter can be enabled for float input data but cannot be enabled for half input data. For details about how to use isReuseSource, see More Examples.

**Table 2** API parameters
Parameter	Input/Output	Description
outputPdGamma	Output	Destination operand, with a shape of [H]. For details about the definition of the LocalTensor data structure, see LocalTensor. The last axis length must be 32-byte aligned. The type is LocalTensor, and TPosition can be VECIN, VECCALC, or VECOUT.
outputPdBeta	Output	Destination operand, with a shape of [H]. For details about the definition of the LocalTensor data structure, see LocalTensor. The last axis length must be 32-byte aligned. The type is LocalTensor, and TPosition can be VECIN, VECCALC, or VECOUT.
resForGamma	Input	Source operand, with a shape of [B, S, H]. For details about the definition of the LocalTensor data structure, see LocalTensor. The data type of resForGamma must be the same as that of the destination operand, and the last axis length must be 32-byte aligned. The LayerNormGrad API needs to be called in advance to obtain the value of resForGamma. The type is LocalTensor, and TPosition can be VECIN, VECCALC, or VECOUT.
inputDy	Input	Source operand, with a shape of [B, S, H]. For details about the definition of the LocalTensor data structure, see LocalTensor. The data type of inputDy must be the same as that of the destination operand, and the last axis length must be 32-byte aligned. The type is LocalTensor, and TPosition can be VECIN, VECCALC, or VECOUT.
sharedTmpBuffer	Input	Shared buffer, which is used to store temporary data generated during internal API computation. This enables developers to manage the sharedTmpBuffer space and reuse the buffer after calling the API, so that the buffer is not repeatedly allocated or deallocated, improving the flexibility and buffer utilization. For details about how to obtain the size of the shared buffer, see LayerNormGradBeta Tiling. The type is LocalTensor, and TPosition can be VECIN, VECCALC, or VECOUT.
tiling	Input	Tiling information required for LayerNormGradBeta computation. For details about how to obtain the tiling information, see LayerNormGradBeta Tiling.

Returns

None

Constraints

For details about the operand address alignment requirements, see General Address Alignment Restrictions.
The tensor space of the source operand and destination operand can be reused.
The input shape must be in ND format.
If the input data does not meet the alignment requirements, you need to pad the data. The padded data should be set to 0 to prevent abnormal values from affecting network computation.
The last axis (H axis) cannot be split.

Examples

For a complete call example, see LayerNormGradBeta operator sample.

// outputPdGamma: output gradient of the gamma parameter, with shape [H]
// outputPdBeta: output gradient of the beta parameter, with shape [H]
// resForGamma: intermediate result output by LayerNormGrad in the previous step, that is, normalizedX x inputDy, with shape [B, S, H]
// inputDy: upstream gradient, with shape [B, S, H]
// tiling: tiling scheduling information, including parameters such as parallel division and block size

// Use the LayerNormGradBeta API to calculate the gradients of gamma and beta.
AscendC::LayerNormGradBeta<T, isReuseSource>(
    outputPdGamma,   // Output: gamma gradient, with shape [H]
    outputPdBeta,    // Output: beta gradient, with shape [H]
    resForGamma,     // Input: intermediate result normalizedX x inputDy, from LayerNormGrad
    inputDy,         // Input: upstream gradient dy, with shape [B, S, H]
    tiling           // Input: tiling information
);

The following is an example:

Input (inputDy, shape: [1, 8, 8]):
[  0.  1.  2.  3.  4.  5.  6.  7.
   8.  9. 10. 11. 12. 13. 14. 15.
  16. 17. 18. 19. 20. 21. 22. 23.
  24. 25. 26. 27. 28. 29. 30. 31.
  32. 33. 34. 35. 36. 37. 38. 39.
  40. 41. 42. 43. 44. 45. 46. 47.
  48. 49. 50. 51. 52. 53. 54. 55.
  56. 57. 58. 59. 60. 61. 62. 63. ]
Input (resForGamma, shape: [1, 8, 8]):
[  0.  1.  2.  3.  4.  5.  6.  7.
   8.  9. 10. 11. 12. 13. 14. 15.
  16. 17. 18. 19. 20. 21. 22. 23.
  24. 25. 26. 27. 28. 29. 30. 31.
  32. 33. 34. 35. 36. 37. 38. 39.
  40. 41. 42. 43. 44. 45. 46. 47.
  48. 49. 50. 51. 52. 53. 54. 55.
  56. 57. 58. 59. 60. 61. 62. 63. ]
Output (outputPdGamma):
[ 8960.  9416.  9888. 10376. 10880. 11400. 11936. 12488.]
Output (outputPdBeta):
[ 224. 232. 240. 248. 256. 264. 272. 280. ]

Parent topic: Normalization