ActivationType

Description

Enumeration of activation function types.

Definition

enum ActivationType : int {
    ACTIVATION_UNDEFINED = 0,
    ACTIVATION_RELU,
    ACTIVATION_GELU,
    ACTIVATION_FAST_GELU,
    ACTIVATION_SWISH,
    ACTIVATION_LOG,
    ACTIVATION_SWIGLU_FORWARD,
    ACTIVATION_SWIGLU_BACKWARD,
    ACTIVATION_SIGMOID,
    ACTIVATION_FASTER_GELU_FORWARD,
    ACTIVATION_MAX
};

Members

Member	Description
ACTIVATION_UNDEFINED	Undefined type
ACTIVATION_RELU	ReLU activation type
ACTIVATION_GELU	GELU activation type
ACTIVATION_FAST_GELU	FAST_GELU activation type Fast version of the GELU activation function. The approximate GELU activation function is executed for each element in a tensor to speed the calculation with higher accuracy.
ACTIVATION_SWISH	SWISH activation type
ACTIVATION_LOG	LOG activation type
ACTIVATION_SWIGLU_FORWARD	SWIGLU_FORWARD activation type SwiGLU forward activation function. For Atlas inference products, only 32-bit aligned data is supported.
ACTIVATION_SWIGLU_BACKWARD	SWIGLU_BACKWARD activation type Backward of the SwiGLU forward activation function, which is used for gradient calculation. Supported only by Atlas A2 training products/Atlas A2 inference products and Atlas A3 inference products/Atlas A3 training products.
ACTIVATION_SIGMOID	SIGMOID activation type
ACTIVATION_FASTER_GELU_FORWARD	FASTER_GELU_FORWARD activation type
ACTIVATION_MAX	Maximum enumerated value, not an activation type

Parent topic: atb/infer_op_params.h