maskType

MaskType Enumeration

Location: include/atb/infer_op_params.h

enum MaskType : int {
    MASK_TYPE_UNDEFINED = 0,             // All-zero mask
    MASK_TYPE_NORM,                      // Causal mask (inverted triangle)
    MASK_TYPE_ALIBI,                     // ALiBi relative position bias
    MASK_TYPE_NORM_COMPRESS,             // Compressed causal mask
    MASK_TYPE_ALIBI_COMPRESS,            // ALiBi-compressed mask
    MASK_TYPE_ALIBI_COMPRESS_SQRT,       // ALiBi-compressed mask + Square root
    MASK_TYPE_ALIBI_COMPRESS_LEFT_ALIGN, // ALiBi-compressed left-aligned mask (only for 910B)
    MASK_TYPE_SLIDING_WINDOW_NORM,       // Sliding window + Inverted triangle
    MASK_TYPE_SLIDING_WINDOW_COMPRESS,   // Compressed sliding window mask
    MASK_TYPE_CAUSAL_MASK,               // Causal mask generated internally by the operator
};

Value	Name	Semantic Abstract
0	UNDEFINED	No mask, equivalent to all zeros.
1	NORM	mask[i,j]=0 if i≥j else -inf; 1 is used as the padding value for int8/bf16, and -inf is used for float16.
2	ALIBI	Linear position bias.
3	NORM_COMPRESS	Block-level inverted triangle; the shape varies by platform ([128, 128], [1, 16, 128, 16], or [2048, 2048]).
4-6	ALIBI_COMPRESS	ALiBi compression + slopes [headNum].
7	SWA_NORM	Sliding window + causal mask. The decoder type can have no mask input, which is generated internally.
8	SWA_COMPRESS	Compressed sliding window mask, dedicated for PA.
9	CAUSAL_MASK	Internally generated; only for PREFIX.

Terms and Mask Input Types

Input Name	Condition	Typical Shape	Description
No mask input	UNDEFINED; CAUSAL_MASK; DECODER+SWA_NORM	-	CAUSAL is generated internally by the operator.
attentionMask	NORM NORM_COMPRESS SWA_* 910B, PA, ALiBi, etc.	See the constraints on mask shape.	Commonly used mask.
slopes	ALIBI_COMPRESS / SQRT / LEFT_ALIGN	[headNum]	ALiBi-compressed mask with added position biases.

Input Name

Condition

Typical Shape

Description

No mask input

UNDEFINED;

CAUSAL_MASK;

DECODER+SWA_NORM

CAUSAL is generated internally by the operator.

attentionMask

NORM

NORM_COMPRESS

SWA_*

910B, PA, ALiBi, etc.

See the constraints on mask shape.

Commonly used mask.

slopes

ALIBI_COMPRESS / SQRT / LEFT_ALIGN

[headNum]

ALiBi-compressed mask with added position biases.

Associated param Fields

Field	Function
maskType	Mask type enumeration.
isTriuMask	Inverted triangle optimization; Cannot be 1 when it is UNDEFINED. Forcibly set to 1 during 310P NORM_COMPRESS construction.
windowSize	> 0: Must work with SLIDING_WINDOW_NORM/COMPRESS.

Field

Function

maskType

Mask type enumeration.

isTriuMask

Inverted triangle optimization;

Cannot be 1 when it is UNDEFINED.

Forcibly set to 1 during 310P NORM_COMPRESS construction.

windowSize

> 0: Must work with SLIDING_WINDOW_NORM/COMPRESS.

Hardware x MaskType Support Matrix

Atlas A2 training products/Atlas A2 inference products, Atlas A3 inference products/Atlas A3 training products

MaskType	PA	ENCODER/DECODER	Remarks
UNDEFINED	√	√	-
NORM	√	√	-
ALIBI	√	√	attentionMask, non-slopes
NORM_COMPRESS	√	√	[128,128]
ALIBI_COMPRESS*	√	-	+ slopes
ALIBI_LEFT_ALIGN	√	-	Atlas A2 training products/Atlas A2 inference products only
SWA_NORM/COMPRESS	√	DECODER(NORM)/PA	-
CAUSAL_MASK	×	-	Only PREFIX, internally generated mask

Atlas training products

Description	Remarks
UNDEFINED, NORM, NORM_COMPRESS (NZ)	PA_ENCODER only

Atlas inference products

MaskType	PA	Remarks
UNDEFINED/NORM/NORM_COMPRESS	√
SWA_*	√	NZ 4D

Atlas 350 accelerator cards

MaskType	PA	Remarks
UNDEFINED	√	PA and BSND only
NORM	√	prefill: masking required; decode: different behavior.
ALIBI	√	pseShift, non-slopes
NORM_COMPRESS	√	ND [2048,2048]

Parent topic: mask