[object Object]

Sequence parallelism (SP) splits the KV cache so that the KV cache saved by each SP rank is different, reducing the graphics memory usage and supporting long sequences.

[object Object]

• The Atlas 800I A2 inference server and Atlas 800I A3 SuperPoD server support this feature.

• Currently, only the W8A8 quantization models of DeepSeek-R1, W4A8 quantization models of DeepSeek-R1, DeepSeek-V3, and DeepSeek-V3.1 support this feature.

• This feature is supported in the prefill-decode disaggregation and prefill-decode co-location scenarios.

• The value of SP must be equal to that of TP.

• In the prefill-decode co-location scenario:

  • This feature can be used together with DP and TP. The product of DP and TP is equal to the value of [object Object].
  • This feature can be used together with CP, TP, and MTP. The product of CP and TP is equal to [object Object].
  • This feature can be used together with asynchronous scheduling and prefix cache, and it can be used in scenarios where MTP equals 1.

• In the prefill-decode disaggregation scenario:

  • SP can be enabled on prefill nodes only. This feature can be used together with DP, TP, and MTP. The product of DP and TP is equal to the value of [object Object].
  • SP can be enabled on prefill nodes only. This feature can be used together with CP, TP, and MTP. The product of CP and TP is equal to the value of [object Object].
  • This feature can be used together with MTP, asynchronous scheduling, and Prefix Cache.

• This feature does not support BF16.

[object Object]

describes the service parameters required for enabling the SP feature.

Table 1 SP parameter [object Object] in [object Object] [object Object][object Object]

[object Object]undefined[object Object]

1. Open the [object Object] file of the server.

   • Installation using the [object Object] package:
   [object Object]
   • Installation using the [object Object] package:
   [object Object]

2. Set serving parameters. Add the "sp" field to the [object Object] file of the server. For details about the parameters, see . For detailed configuration of the [object Object] file: refer to "Cluster Service Deployment" > "PD Disaggregation" in MindIE Motor Development Guide for PD disaggregation and refer to "Configuring MindIE" > "Configuring Server" > "Multi-Node Inference" in MindIE Installation Guide for PD co-location.

   [object Object]

3. Start the service.

   • Installation using the [object Object] package:
   [object Object]
   • Installation using the [object Object] package:
   [object Object]