ActivationType

Description

Enumeration of activation function types.

Definition

enum ActivationType : int {
    ACTIVATION_UNDEFINED = 0,
    ACTIVATION_RELU,
    ACTIVATION_GELU,
    ACTIVATION_FAST_GELU,
    ACTIVATION_SWISH,
    ACTIVATION_LOG,
    ACTIVATION_SWIGLU_FORWARD,
    ACTIVATION_SWIGLU_BACKWARD,
    ACTIVATION_SIGMOID,
    ACTIVATION_FASTER_GELU_FORWARD,
    ACTIVATION_MAX,
};

Members

Member	Description
ACTIVATION_UNDEFINED	Undefined type
ACTIVATION_RELU	RELU activation type
ACTIVATION_GELU	GELU activation type
ACTIVATION_FAST_GELU	FAST_GELU activation type Fast version of the GELU activation function. The approximate GELU activation function is executed for each element in a tensor to speed the calculation with higher accuracy.
ACTIVATION_SWISH	SWISH activation type
ACTIVATION_LOG	LOG activation type
ACTIVATION_SWIGLU_FORWARD	SWIGLU_FORWARD activation type. SwiGLU forward activation function. For the Atlas inference products, only 32-bit aligned data is supported.
ACTIVATION_SWIGLU_BACKWARD	SWIGLU_BACKWARD activation type. Backward of the SwigLU forward activation function, which is used for gradient calculation. It is supported only by the Atlas A2 inference products.
ACTIVATION_SIGMOID	SIGMOID activation type
ACTIVATION_FASTER_GELU_FORWARD	FASTER_GELU_FORWARD activation type
ACTIVATION_MAX	Maximum enumerated value, inactive type

Parent topic: atb/infer_op_params.h