Operator Context
- When calcType is set to PA_ENCODER, the full phase of paged attention is used, as shown in the following figure.
Figure 1 Location of the full phase of paged attention in the model
- In other cases, the full/incremental phase of the traditional flash attention is used, as shown in the following figure.
Figure 2 Location of the flash attention in the model
Parent topic: SelfAttentionOperation