Context parallelism (CP) performs parallel computing for the self-attention module in the sequence dimension. CP splits long sequences in the context dimension, allocates the sequences to different devices for parallel processing, and reduces the response time of the first token. The CP implementation includes the following:
- Each device calculates its own attention, and devices transfer KV values in ring mode to obtain the result of the block-based computation. The overall principle is similar to ring-attention.
- The flash-attention 2 algorithm is used to perform block-based computation and correct the result.
The Atlas 800I A2 inference server and Atlas 800I A3 SuperPoD server support this feature.
Currently, only the W8A8 quantization models of DeepSeek-R1, W4A8 quantization models of DeepSeek-R1, DeepSeek-V3, and DeepSeek-V3.1 support this feature.
Currently, CP cannot be enabled independently. To enable CP, sequence parallelism (SP) must be enabled at the same time.
This feature is supported in the prefill-decode disaggregation and prefill-decode co-location scenarios.
In the prefill-decode co-location scenario:
- This feature can be used together with SP and tensor parallelism (TP). When CP is enabled, the value of DP must be 1, the value of SP must be equal to that of TP, and the product of CP, DP, and TP must be equal to the value of
[object Object]. - This feature supports combined use with MTP=1, asynchronous scheduling, and Prefix Cache.
- This feature can be used together with SP and tensor parallelism (TP). When CP is enabled, the value of DP must be 1, the value of SP must be equal to that of TP, and the product of CP, DP, and TP must be equal to the value of
In the prefill-decode disaggregation scenario:
- CP can be enabled on prefill nodes only. This feature can be used together with SP, TP, and MTP. When CP is enabled, the value of DP must be 1, the value of SP must be equal to that of TP, and the product of CP, DP, and TP must be equal to the value of
[object Object]. - This feature can be used together with MTP, asynchronous scheduling, and Prefix Cache.
- CP can be enabled on prefill nodes only. This feature can be used together with SP, TP, and MTP. When CP is enabled, the value of DP must be 1, the value of SP must be equal to that of TP, and the product of CP, DP, and TP must be equal to the value of
This feature does not support BF16.
[Table 1](#ModelConfig parameters) describes the serving parameters required for enabling the CP feature.
Table 1 ModelConfig in ModelDeployConfig [object Object][object Object]
Open the
[object Object]file of the server.- Installation using the
[object Object]package:
[object Object]- Installation using the
[object Object]package:
[object Object]- Installation using the
Set serving parameters. Add the
[object Object]field (the following information in bold) to the[object Object]file of the server. For details about the parameters, see . For details about the serving parameters, see . The following is an example of parameter settings.[object Object]Start the service.
Installation using the
[object Object]package:[object Object]Installation using the
[object Object]package:[object Object]