Swish

Applicability

Product	Supported
Atlas 350 Accelerator Card	√
Atlas A3 training product/Atlas A3 inference product	√
Atlas A2 training product/Atlas A2 inference product	√
Atlas 200I/500 A2 inference product	x
Atlas inference product AI Core	√
Atlas inference product Vector Core	x
Atlas training product	x

Function Usage

In neural networks, Swish is an important activation function. The following is the formula, where β is a constant:

$\text{[math]}$

Prototype

template <typename T, bool isReuseSource = false>
__aicore__ inline void Swish(const LocalTensor<T>& dstLocal, const LocalTensor<T>& srcLocal, uint32_t dataSize, const T scalarValue)

Parameters

**Table 1** Template parameters
Parameter	Description
T	Data type of the operand. For the Atlas 350 Accelerator Card, the supported data types are half and float. For the Atlas A3 training product/Atlas A3 inference product, the supported data types are half and float. For the Atlas A2 training product/Atlas A2 inference product, the supported data types are half and float. For the Atlas inference product AI Core, the supported data types are half and float.
isReuseSource	Whether the source operand can be modified. This parameter is reserved. Pass the default value false.

**Table 2** API parameters
Parameter	Input/Output	Description
dstLocal	Output	Destination operand. The type is LocalTensor, and TPosition can be VECIN, VECCALC, or VECOUT.
srcLocal	Input	Source operand. The source operand must have the same data type as the destination operand. The type is LocalTensor, and TPosition can be VECIN, VECCALC, or VECOUT.
dataSize	Input	Number of elements involved in the computation.
scalarValue	Input	β in the activation function. The supported data types are half and float. The data type of β must be the same as that of the source and destination operands.

Returns

None

Constraints

For details about the alignment requirements of the operand address offset, see General Description and Restrictions.
The source operand address must not overlap the destination operand address.
Currently, only the ND format is supported.

Examples

For a complete operator sample, see swish operator sample.

// dstLocal: tensor for storing the Swish computation result
// srcLocal: tensor for storing the Swish computation input
// scalarValue: β in the activation function

// The number of elements involved in the computation is 32, and the value of the β parameter is -1.702.
AscendC::Swish(dstLocal, srcLocal, 32, -1.702);

Result example:

Input (srcLocal):
[-4.         -3.7419355  -3.483871   -3.2258065  -2.967742   -2.7096775  -2.451613   -2.1935484
 -1.9354838  -1.6774193  -1.4193548  -1.1612903  -0.9032258  -0.6451613  -0.38709676 -0.12903225
  0.12903225  0.38709676  0.6451613   0.9032258   1.1612903   1.4193548   1.6774193   1.9354838
  2.1935484   2.451613    2.7096775   2.967742    3.2258065   3.483871    3.7419355   4.        ]
Output (dstLocal):
[-0.00441472 -0.00640367 -0.00924141 -0.01325663 -0.01887952 -0.02665107 -0.03720944 -0.05122496
 -0.06923404 -0.09129371 -0.11635891 -0.14131825 -0.15980731 -0.16135658 -0.13200021 -0.05746017
  0.07157208  0.25509655  0.48380467  0.74341846  1.019972    1.3029958   1.5861256   1.8662498
  2.1423235   2.4144034   2.6830263   2.9488626   3.2125497   3.4746296   3.735532    3.995585  ]

Parent topic: Swish APIs