Computational Graphs
- The OP internal graphs of the ATB vary according to the parameter settings. When calcType is set to PA_ENCODER, the computational graph is shown in the following figure.
Figure 1 Computational graph of the full phase of paged attention
- When calcType is not set to PA_ENCODER and kvCacheCfg is set to K_CACHE_V_CACHE, the computational graph is shown in the following figure.
Figure 2 Computational graph when flash attention kvCacheCfg is set to K_CACHE_V_CACHE
- When calcType is not set to PA_ENCODER and kvCacheCfg is set to K_BYPASS_V_BYPASS, the computational graph is shown in the following figure.
Figure 3 Computational graph when flash attention kvCacheCfg is set to K_BYPASS_V_BYPASS
Parent topic: SelfAttentionOperation