Drive I/O Configuration
Principle: NVMe SSDs provide faster and more efficient read and write performance than standard SATA SSDs. The main differences lie in transmission interfaces and protocols. NVMe SSDs use the high-speed PCIe interface and the purpose-built NVMe protocol. By connecting directly to the CPU, they achieve significantly lower latency and deliver read/write speeds multiple times faster than traditional SATA SSDs.
Optimization configuration: You are advised to use NVMe SSDs instead of standard SSDs as system disks.
Recommended scenario: You are advised to use NVMe SSDs instead of standard SSDs as system disks to improve the read and write performance of disks. In inference scenarios, this helps improve the model performance.
In the experiment, the Atlas 800T A2 is used. The NVMe SSD and standard SSD are used as the system disks to install the openEuler 22.03 SP4 OS to study the impact of different system disks on the model inference performance. Software and hardware configurations vary. The data listed in the following table is for reference only.
Model |
Concurrency |
Input Length |
Experiment No. |
Common SSD System Disk (Tokens/s) |
NVMe System Disk (Tokens/s) |
Performance Gains (%) |
|---|---|---|---|---|---|---|
Llama-7B |
8 |
128 |
Experiment 1 |
75.0003 |
75.4373 |
0.58 |
Experiment 2 |
75.1019 |
75.3953 |
0.39 |
|||
Experiment 3 |
75.1043 |
75.4051 |
0.40 |
|||
Average value |
75.0688 |
75.4126 |
0.46 |
|||
8 |
256 |
Experiment 1 |
76.3483 |
76.5359 |
0.25 |
|
Experiment 2 |
76.1594 |
76.5362 |
0.49 |
|||
Experiment 3 |
76.2051 |
77.0832 |
1.15 |
|||
Average value |
76.2376 |
76.7184 |
0.63 |
|||
Qwen2-7B |
8 |
128 |
Experiment 1 |
82.2465 |
83.5893 |
1.63 |
Experiment 2 |
82.9202 |
83.4479 |
0.64 |
|||
Experiment 3 |
82.8264 |
83.3766 |
0.66 |
|||
Average value |
82.6644 |
83.4713 |
0.98 |
|||
8 |
256 |
Experiment 1 |
83.7558 |
84.7990 |
1.25 |
|
Experiment 2 |
84.1594 |
84.7267 |
0.67 |
|||
Experiment 3 |
84.2340 |
84.8510 |
0.73 |
|||
Average value |
84.0497 |
84.7922 |
0.88 |