During weight loading, MindIE fully loads the weight files in [object Object] format by default, shards the weights in memory based on the parallelism policy, and then transfers the weights to the NPU in host-to-device (H2D) mode. For large-scale parameter models such as DeepSeek, an offline weight sharding policy can be used to reduce the weight loading time. Weights are sharded in advance based on the runtime parallelism policy and stored in tmpfs to implement efficient loading.

• Only the DeepSeek-R1 and DeepSeek-V3 models support this feature.
• The configuration used for offline weight sharding must be consistent with the configuration used during model inference.
• This feature is supported only in the Atlas 800I A2 inference server two-node cluster scenario and the Atlas 800I A3 SuperPoD server single-node scenario.
• This feature cannot be enabled together with the shared expert and routed expert merging feature.
• This feature cannot be enabled together with the dynamic load balancing feature.

The following uses the Atlas 800I A3 SuperPoD server single-node scenario as an example. You can use the following script to shard the weights.

The following uses dual Atlas 800I A2 inference servers as an example. You can use the following script to shard weights.

Weight directory structure after sharding:

[object Object]

The online serving inference scenario is used as an example.

1. Open the [object Object] file of the server.

   • Installation using the [object Object] package:
   [object Object]
   • Installation using the [object Object] package:
   [object Object]

2. Set serving parameters. Change the model weight path to the path where the sharded weight files are stored. For details about the serving parameters, see . The following is an example of parameter configuration:

   [object Object]