Gelu

Applicability

Product	Supported
Atlas A3 training products/Atlas A3 inference products	√
Atlas A2 training products/Atlas A2 inference products	√
Atlas 200I/500 A2 inference products	x
Atlas inference product's AI Core	√
Atlas inference product's Vector Core	x
Atlas training products	x

Function

In neural networks, GELU is an important activation function, which is inspired by ReLU and Dropout. Specifically, random regularization is introduced during activation. Below is the formula.

$\text{[math]}$

$\text{[math]}$ is simplified to obtain $\text{[math]}$

Prototype

Allocate the temporary space through the API framework.

template <typename T, bool highPrecision = false, bool highPerformance = false>
__aicore__ inline void Gelu(const LocalTensor<T>& dstLocal, const LocalTensor<T>& srcLocal, const uint32_t dataSize)

Pass to the temporary space through the sharedTmpBuffer input parameter.

template <typename T, bool highPrecision = false, bool highPerformance = false>
__aicore__ inline void Gelu(const LocalTensor<T>& dstLocal, const LocalTensor<T>& srcLocal, const LocalTensor<uint8_t>& sharedTmpBuffer, const uint32_t dataSize)

Parameters

**Table 1** Template parameters
Parameter	Description
T	Data type of the operand. For the Atlas A3 training products/Atlas A3 inference products, the supported data types are half and float. For the Atlas A2 training products/Atlas A2 inference products, the supported data types are half and float. For the Atlas inference product's AI Core, the supported data types are half and float.
highPrecision	Whether to enable the high-precision mode to improve the computation accuracy. The default value is false, indicating that the high-precision mode is disabled. Note: The high-precision mode takes effect only when it is enabled for the half type. The value of this parameter does not affect the API precision and performance of the float type.
highPerformance	Whether to enable the high-performance mode to improve the computation efficiency. The default value is false, indicating that the high-performance mode is disabled. Note: Enabling the high-performance mode can result in a decrease in precision when compared to disabling the high-precision and high-performance modes by default. Enabling both the high-precision and high-performance modes may result in a decrease in performance when compared to enabling only the high-performance mode.

**Table 2** API parameters
Parameter	Input/Output	Description
dstLocal	Output	Destination operand. The type is LocalTensor, and the supported TPosition is VECIN, VECCALC, or VECOUT.
srcLocal	Input	Source operand. The type is LocalTensor, and the supported TPosition is VECIN, VECCALC, or VECOUT. The source operand must have the same data type as the destination operand.
sharedTmpBuffer	Input	Temporary buffer. The type is LocalTensor, and the supported TPosition is VECIN, VECCALC, or VECOUT. This parameter is used to store intermediate variables during complex computation and is provided by developers. For details about how to obtain the temporary space size (BufferSize), see GetGeluMaxMinTmpSize.
dataSize	Input	Number of elements involved in the computation.

Returns

None

Restrictions

The tensor space of the source operand and destination operand can be reused.
The address of sharedTmpBuffer must not overlap that of the source or destination operand.
For details about the operand address alignment requirements, see General Address Alignment Restrictions.
The input shape must be in ND format.

Example

#include "kernel_operator.h"

template <typename srcType>
class KernelGelu
{
public:
    __aicore__ inline KernelGelu() {}
    __aicore__ inline void Init(GM_ADDR src_gm, GM_ADDR dst_gm, uint32_t inputSize)
    {
        dataSize = inputSize;
        src_global.SetGlobalBuffer(reinterpret_cast<__gm__ srcType *>(src_gm), dataSize);
        dst_global.SetGlobalBuffer(reinterpret_cast<__gm__ srcType *>(dst_gm), dataSize);
        pipe.InitBuffer(inQueueX, 1, dataSize * sizeof(srcType));
        pipe.InitBuffer(outQueue, 1, dataSize * sizeof(srcType));
    }
    __aicore__ inline void Process()
    {
        CopyIn();
        Compute();
        CopyOut();
    }

private:
    __aicore__ inline void CopyIn()
    {
        AscendC::LocalTensor<srcType> srcLocal = inQueueX.AllocTensor<srcType>();
        AscendC::DataCopy(srcLocal, src_global, dataSize);
        inQueueX.EnQue(srcLocal);
    }
    __aicore__ inline void Compute()
    {
        AscendC::LocalTensor<srcType> dstLocal = outQueue.AllocTensor<srcType>();
        AscendC::LocalTensor<srcType> srcLocal = inQueueX.DeQue<srcType>();
        AscendC::Gelu(dstLocal, srcLocal, dataSize);
        // AscendC::Gelu<srcType, true, false>(dstLocal, srcLocal, dataSize);
        // AscendC::Gelu<srcType, false, true>(dstLocal, srcLocal, dataSize);
        outQueue.EnQue<srcType>(dstLocal);
        inQueueX.FreeTensor(srcLocal);
    }
    __aicore__ inline void CopyOut()
    {
        AscendC::LocalTensor<srcType> dstLocal = outQueue.DeQue<srcType>();
        AscendC::DataCopy(dst_global, dstLocal, dataSize);
        outQueue.FreeTensor(dstLocal);
    }

private:
    AscendC::GlobalTensor<srcType> src_global;
    AscendC::GlobalTensor<srcType> dst_global;
    AscendC::TPipe pipe;
    AscendC::TQue<AscendC::TPosition::VECIN, 1> inQueueX;
    AscendC::TQue<AscendC::TPosition::VECOUT, 1> outQueue;
    uint32_t dataSize = 0;
};

template <typename dataType>
__aicore__ void kernel_Gelu_operator(GM_ADDR src_gm, GM_ADDR dst_gm, uint32_t dataSize)
{
    KernelGelu<dataType> op;
    op.Init(src_gm, dst_gm, dataSize);
    op.Process();
}

Result example:

Input data (srcLocal):
[-1.251   1.074  -6.137  -9.67   -5.066  -9.44   -3.588  -5.758  -7.484
 -5.35   -9.62   -4.33   -6.66   -3.732   0.0841 -8.59   -6.3    -4.62
 -3.059  -8.34   -8.24   -7.617  -7.93   -3.592  -3.268  -5.406  -9.49
  5.633  -5.3    -9.36   -6.715  -5.727 ]
Output data (dstLocal):
[-0.1411  0.916  -0.     -0.     -0.     -0.     -0.     -0.     -0.
 -0.     -0.     -0.     -0.     -0.      0.0486 -0.     -0.     -0.
 -0.     -0.     -0.     -0.     -0.     -0.     -0.     -0.     -0.
  5.633  -0.     -0.     -0.     -0.    ]

Parent topic: Gelu APIs