[root@ZBHF-ZSCluster1-AT800-10 ~]# benchmark \
--DatasetPath "/root/gsm8k/" \
--DatasetType gsm8k \
--ModelName deepseek-r1 \
--ModelPath "/root/.cache/huggingface/DeepSeek-R1-bf16-hfd-w8a8" \
--TestType client \
--Http "http://172.20.112.20:1205" \
--ManagementHttp "http://172.20.112.20:1206" \
--Concurrency 128 \
--TaskKind stream \
--Tokenizer False \
--MaxOutputLen 512
/usr/local/lib/python3.11/site-packages/mindiebenchmark/benchmark_run.py:21: MonkeyPatchWarning: Monkey-patching ssl after ssl has already been imported may lead to errors, including RecursionError on Python 3.6. It may also silently
lead to incorrect behaviour on Python 3.7. Please monkey-patch earlier. See https://github.com/gevent/gevent/issues/1016. Modules that had direct imports (NOT patched): ['urllib3.util (/usr/local/lib/python3.11/site-packages/urllib3/u
til/__init__.py)', 'urllib3.util.ssl_ (/usr/local/lib/python3.11/site-packages/urllib3/util/ssl_.py)'].
monkey.patch_all(thread=False)
Please execute command `export MINDIE_LOG_TO_STDOUT="benchmark:1; client:1"` if log message need to be displayed on screen next time.
[2025-03-19 18:06:28.424+08:00] [25173] [281473893255904] [benchmark] [INFO] [benchmark_run.py:93] Running Task.
[2025-03-19 18:06:28.425+08:00] [25173] [281473893255904] [benchmark] [INFO] [benchmark_run.py:103] [UserId]: root, [Ip]: localhost, [Model]: deepseek-r1, [TaskType]: Performance, [TestType]: client, [DatasetType]: gsm8k, [Http]: http
://172.20.112.20:1205
[2025-03-19 18:06:28.425+08:00] [25173] [281473893255904] [benchmark] [INFO] [benchmark_run.py:46] The Benchmark init instance_path: ./instance
Benchmark task is running, please wait...
[2025-03-19 18:06:28.884+08:00] [25173] [281462532665760] [benchmark] [INFO] [base_loader.py:103] Reading dataset...
[2025-03-19 18:06:28.885+08:00] [25173] [281462532665760] [benchmark] [INFO] [base_loader.py:112] Finished loading dataset...
[2025-03-19 18:06:28.885+08:00] [25173] [281473893255904] [benchmark] [INFO] [benchmarker.py:72] MULTI MACHINE is running
[2025-03-19 18:06:28.886+08:00] [25173] [281473893255904] [benchmark] [INFO] [benchmarker.py:76] Multi machine is running in [slave] machine node
[2025-03-19 18:06:28.886+08:00] [25173] [281473893255904] [benchmark] [INFO] [benchmarker.py:44] Starting benchmark...
[2025-03-19 18:06:28.886+08:00] [25173] [281473893255904] [benchmark] [INFO] [benchmarker.py:122] Starting run mindie-client inference
[2025-03-19 18:06:28.888+08:00] [25173] [281473893255904] [client] [WARN] [base_client.py:145] [MIE02W000007] SSL authentication is disabled!
[2025-03-19 18:06:28.888+08:00] [25173] [281473893255904] [benchmark] [INFO] [base_api.py:77] DoSampling is disabled
[2025-03-19 18:06:28.889+08:00] [25173] [281473893255904] [benchmark] [INFO] [base_api.py:82] Sampling params is {}.
[2025-03-19 18:06:28.889+08:00] [25173] [281473893255904] [benchmark] [INFO] [base_api.py:89] Client init finished. Gevent pool size is 128, max connection for mindie-client is 1000.
[2025-03-19 18:06:28.889+08:00] [25173] [281473893255904] [benchmark] [INFO] [base_api.py:164] Reset states...
[2025-03-19 18:06:28.889+08:00] [25173] [281473893255904] [benchmark] [WARN] [base_api.py:173] [MIE01W000102] Do not check the status of server because management port is disabled
[2025-03-19 18:06:28.898+08:00] [25173] [281473893255904] [client] [ERROR] [utils.py:47] Request failed and the reason is HTTPConnectionPool(host='172.20.112.20', port=1206): Max retries exceeded with url: /v2/health/ready (Caused by
NewConnectionError(': Failed to establish a new connection: [Errno 111] Connection refused')).
[2025-03-19 18:06:28.898+08:00] [25173] [281473893255904] [client] [WARN] [base_client.py:145] [MIE02W000007] SSL authentication is disabled!
[2025-03-19 18:06:28.898+08:00] [25173] [281473893255904] [benchmark] [INFO] [base_api.py:77] DoSampling is disabled
[2025-03-19 18:06:28.898+08:00] [25173] [281473893255904] [benchmark] [INFO] [base_api.py:82] Sampling params is {}.
[2025-03-19 18:06:28.899+08:00] [25173] [281473893255904] [benchmark] [INFO] [base_api.py:89] Client init finished. Gevent pool size is 128, max connection for mindie-client is 1000.
[2025-03-19 18:06:28.899+08:00] [25173] [281473893255904] [benchmark] [INFO] [base_api.py:164] Reset states...
[2025-03-19 18:06:28.899+08:00] [25173] [281473893255904] [benchmark] [WARN] [base_api.py:173] [MIE01W000102] Do not check the status of server because management port is disabled
[2025-03-19 18:06:28.899+08:00] [25173] [281473893255904] [benchmark] [INFO] [base_api.py:318] Warm up start...
[2025-03-19 18:06:28.914+08:00] [25173] [281473893255904] [client] [ERROR] [utils.py:47] [MIE02E000002] The prompt should be a string or a list!
[2025-03-19 18:06:28.915+08:00] [25173] [281473893255904] [benchmark] [ERROR] [base_api.py:348] [MIE01E000108] Warm up request failed, request warm_up_0_1cc83c69593647efb4c589c32f5e55d3 occurs error with message [MIE02E000002] The pro
mpt should be a string or a list!.
[2025-03-19 18:06:28.915+08:00] [25173] [281473893255904] [client] [ERROR] [utils.py:47] [MIE02E000002] The prompt should be a string or a list!
[2025-03-19 18:06:28.915+08:00] [25173] [281473893255904] [benchmark] [ERROR] [base_api.py:348] [MIE01E000108] Warm up request failed, request warm_up_1_ba7300defe7a4ada8a13ea8ac33aa586 occurs error with message [MIE02E000002] The pro
mpt should be a string or a list!.
[2025-03-19 18:06:28.915+08:00] [25173] [281473893255904] [client] [ERROR] [utils.py:47] [MIE02E000002] The prompt should be a string or a list!
[2025-03-19 18:06:28.916+08:00] [25173] [281473893255904] [benchmark] [ERROR] [base_api.py:348] [MIE01E000108] Warm up request failed, request warm_up_2_cc83b004e6d040f5b3d20ccf383a7c25 occurs error with message [MIE02E000002] The pro
mpt should be a string or a list!.
[2025-03-19 18:06:28.916+08:00] [25173] [281473893255904] [client] [ERROR] [utils.py:47] [MIE02E000002] The prompt should be a string or a list!
[2025-03-19 18:06:28.916+08:00] [25173] [281473893255904] [benchmark] [ERROR] [base_api.py:348] [MIE01E000108] Warm up request failed, request warm_up_3_8fa19475bb6745bbb6652b5d9a355d3d occurs error with message [MIE02E000002] The pro
mpt should be a string or a list!.
[2025-03-19 18:06:28.916+08:00] [25173] [281473893255904] [client] [ERROR] [utils.py:47] [MIE02E000002] The prompt should be a string or a list!
[2025-03-19 18:06:28.917+08:00] [25173] [281473893255904] [benchmark] [ERROR] [base_api.py:348] [MIE01E000108] Warm up request failed, request warm_up_4_119ba95738e243c7bd0c7149bb1de2c4 occurs error with message [MIE02E000002] The pro
mpt should be a string or a list!.
[2025-03-19 18:06:28.917+08:00] [25173] [281473893255904] [client] [ERROR] [utils.py:47] [MIE02E000002] The prompt should be a string or a list!
[2025-03-19 18:06:28.917+08:00] [25173] [281473893255904] [benchmark] [ERROR] [base_api.py:348] [MIE01E000108] Warm up request failed, request warm_up_5_bc8b2ee814bf4c45ab11d861fc268999 occurs error with message [MIE02E000002] The pro
mpt should be a string or a list!.
[2025-03-19 18:06:28.917+08:00] [25173] [281473893255904] [client] [ERROR] [utils.py:47] [MIE02E000002] The prompt should be a string or a list!
[2025-03-19 18:06:28.918+08:00] [25173] [281473893255904] [benchmark] [ERROR] [base_api.py:348] [MIE01E000108] Warm up request failed, request warm_up_6_ca4a3523da294b09b9a11105408b5cc3 occurs error with message [MIE02E000002] The pro
mpt should be a string or a list!.
[2025-03-19 18:06:28.918+08:00] [25173] [281473893255904] [client] [ERROR] [utils.py:47] [MIE02E000002] The prompt should be a string or a list!
[2025-03-19 18:06:28.918+08:00] [25173] [281473893255904] [benchmark] [ERROR] [base_api.py:348] [MIE01E000108] Warm up request failed, request warm_up_7_fd060518395c40af91f641ff4ef99203 occurs error with message [MIE02E000002] The pro
mpt should be a string or a list!.
[2025-03-19 18:06:28.918+08:00] [25173] [281473893255904] [client] [ERROR] [utils.py:47] [MIE02E000002] The prompt should be a string or a list!
[2025-03-19 18:06:28.919+08:00] [25173] [281473893255904] [benchmark] [ERROR] [base_api.py:348] [MIE01E000108] Warm up request failed, request warm_up_8_a6ea1226913a48bfb1f5e4ad7c718470 occurs error with message [MIE02E000002] The pro
mpt should be a string or a list!.
[2025-03-19 18:06:28.919+08:00] [25173] [281473893255904] [client] [ERROR] [utils.py:47] [MIE02E000002] The prompt should be a string or a list!
[2025-03-19 18:06:28.919+08:00] [25173] [281473893255904] [benchmark] [ERROR] [base_api.py:348] [MIE01E000108] Warm up request failed, request warm_up_9_5ce0664edea349c3ad838be4ef494e90 occurs error with message [MIE02E000002] The pro
mpt should be a string or a list!.
[2025-03-19 18:06:28.919+08:00] [25173] [281473893255904] [benchmark] [INFO] [base_api.py:335] Warm up finished. Handling 10 requests, cost warm up time 0.006 seconds
[2025-03-19 18:06:28.920+08:00] [25173] [281473893255904] [benchmark] [INFO] [base_api.py:295] Handling 0 requests, failed 0
[2025-03-19 18:06:28.920+08:00] [25173] [281473893255904] [benchmark] [ERROR] [base_api.py:237] [MIE01E000005] All requests failed, please check dataset path or other settings.
[2025-03-19 18:06:28.920+08:00] [25173] [281473893255904] [benchmark] [INFO] [output.py:280] calc_common_metrics start time: 1742378788.8863451 !
[2025-03-19 18:06:28.921+08:00] [25173] [281473893255904] [benchmark] [WARN] [output.py:282] [MIE01W000005] Empty response numbers: 0 !
[2025-03-19 18:06:28.924+08:00] [25173] [281473893255904] [benchmark] [INFO] [output.py:115]
The BenchMark test performance metric result is:
+---------------------+-----------+-----------+-----------+-----------+-----------+-----------+-----------+---+
| Metric | average | max | min | P75 | P90 | SLO_P90 | P99 | N |
+---------------------+-----------+-----------+-----------+-----------+-----------+-----------+-----------+---+
| FirstTokenTime | 0 ms | 0 ms | 0 ms | 0 ms | 0 ms | 0 ms | 0 ms | 0 |
| DecodeTime | 0 ms | 0 ms | 0 ms | 0 ms | 0 ms | 0 ms | 0 ms | 0 |
| LastDecodeTime | 0 ms | 0 ms | 0 ms | 0 ms | 0 ms | 0 ms | 0 ms | 0 |
| MaxDecodeTime | 0 ms | 0 ms | 0 ms | 0 ms | 0 ms | 0 ms | 0 ms | 0 |
| GenerateTime | 0 ms | 0 ms | 0 ms | 0 ms | 0 ms | 0 ms | 0 ms | 0 |
| InputTokens | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 |
| GeneratedTokens | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 |
| GeneratedTokenSpeed | 0 token/s | 0 token/s | 0 token/s | 0 token/s | 0 token/s | 0 token/s | 0 token/s | 0 |
| GeneratedCharacters | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 |
| Tokenizer | 0 ms | 0 ms | 0 ms | 0 ms | 0 ms | 0 ms | 0 ms | 0 |
| Detokenizer | 0 ms | 0 ms | 0 ms | 0 ms | 0 ms | 0 ms | 0 ms | 0 |
| CharactersPerToken | 0 | / | / | / | / | / | / | 0 |
| PostProcessingTime | 0 ms | 0 ms | 0 ms | 0 ms | 0 ms | 0 ms | 0 ms | 0 |
| ForwardTime | 0 ms | 0 ms | 0 ms | 0 ms | 0 ms | 0 ms | 0 ms | 0 |
+---------------------+-----------+-----------+-----------+-----------+-----------+-----------+-----------+---+
[2025-03-19 18:06:28.926+08:00] [25173] [281473893255904] [benchmark] [INFO] [output.py:121]
The BenchMark test common metric result is:
+------------------------+---------------------+
| Common Metric | Value |
+------------------------+---------------------+
| CurrentTime | 2025-03-19 18:06:28 |
| TimeElapsed | 0 s |
| DataSource | /root/gsm8k/ |
| Failed | 0( 0 ) |
| Returned | 0( 0 ) |
| Total | 0[ 0 ] |
| Concurrency | 128 |
| ModelName | deepseek-r1 |
| lpct | 0 ms |
| Throughput | 0 req/s |
| GenerateSpeedPerClient | 0 token/s |
| accuracy | / |
+------------------------+---------------------+
[2025-03-19 18:06:28.926+08:00] [25173] [281473893255904] [benchmark] [WARN] [visual_utils.py:130] [MIE01W000004] No data to draw histogram: Frequency_Histogram_of_Decoder_Batch_Size_None.png, return. Please verify which mode has the
data to plot
[2025-03-19 18:06:28.926+08:00] [25173] [281473893255904] [benchmark] [WARN] [visual_utils.py:130] [MIE01W000004] No data to draw histogram: Frequency_Histogram_of_Prefill_Batch_Size_None.png, return. Please verify which mode has the
data to plot
[2025-03-19 18:06:28.927+08:00] [25173] [281473893255904] [benchmark] [WARN] [visual_utils.py:130] [MIE01W000004] No data to draw histogram: Frequency_Histogram_of_Decode_Token_Latencies_None.png, return. Please verify which mode has
the data to plot
[2025-03-19 18:06:28.927+08:00] [25173] [281473893255904] [benchmark] [WARN] [visual_utils.py:130] [MIE01W000004] No data to draw histogram: Frequency_Histogram_of_Prefill_Token_Latencies_None.png, return. Please verify which mode has
the data to plot
[2025-03-19 18:06:28.927+08:00] [25173] [281473893255904] [benchmark] [WARN] [visual_utils.py:130] [MIE01W000004] No data to draw histogram: Frequency_Histogram_of_Queue_Wait_Time_None.png, return. Please verify which mode has the dat
a to plot
[2025-03-19 18:06:28.927+08:00] [25173] [281473893255904] [benchmark] [WARN] [visual_utils.py:130] [MIE01W000004] No data to draw histogram: Frequency_Histogram_of_Tokenizer_Time_None.png, return. Please verify which mode has the data
to plot
[2025-03-19 18:06:28.928+08:00] [25173] [281473893255904] [benchmark] [WARN] [visual_utils.py:130] [MIE01W000004] No data to draw histogram: Frequency_Histogram_of_Detokenizer_Time_None.png, return. Please verify which mode has the da
ta to plot
[2025-03-19 18:06:28.928+08:00] [25173] [281473893255904] [benchmark] [WARN] [visual_utils.py:130] [MIE01W000004] No data to draw histogram: Frequency_Histogram_of_Post_Processing_Latencies_None.png, return. Please verify which mode h
as the data to plot
[2025-03-19 18:06:28.928+08:00] [25173] [281473893255904] [benchmark] [WARN] [visual_utils.py:178] [MIE01W000004] No data to plot: Scatter_Plot_of_Decoder_Batch_Size_vs_Decode_Token_Latencies_None.png, return.Please verify which mode
has the data to plot
[2025-03-19 18:06:28.928+08:00] [25173] [281473893255904] [benchmark] [WARN] [visual_utils.py:178] [MIE01W000004] No data to plot: Scatter_Plot_of_Prefill_Sequence_Length_vs_Prefill_Token_Latencies_None.png, return.Please verify which
mode has the data to plot
[2025-03-19 18:06:28.929+08:00] [25173] [281473893255904] [benchmark] [WARN] [visual_utils.py:178] [MIE01W000004] No data to plot: Scatter_Plot_of_Prefill_Sequence_Length_vs_Prefill_Token_Per_Second_None.png, return.Please verify whic
h mode has the data to plot
Benchmark task completed successfully!
点赞
收藏
回复
分享
举报
已解决
[root@ZBHF-ZSCluster1-AT800-10 ~]# benchmark \
--DatasetPath "/root/gsm8k/" \
--DatasetType gsm8k \
--ModelName deepseek-r1 \
--ModelPath "/root/.cache/huggingface/DeepSeek-R1-bf16-hfd-w8a8" \
--TestType client \
--Http "http://172.20.112.20:1205" \
--ManagementHttp "http://172.20.112.20:1206" \
--Concurrency 128 \
--TaskKind stream \
--Tokenizer False \
--MaxOutputLen 512
/usr/local/lib/python3.11/site-packages/mindiebenchmark/benchmark_run.py:21: MonkeyPatchWarning: Monkey-patching ssl after ssl has already been imported may lead to errors, including RecursionError on Python 3.6. It may also silently
lead to incorrect behaviour on Python 3.7. Please monkey-patch earlier. See https://github.com/gevent/gevent/issues/1016. Modules that had direct imports (NOT patched): ['urllib3.util (/usr/local/lib/python3.11/site-packages/urllib3/u
til/__init__.py)', 'urllib3.util.ssl_ (/usr/local/lib/python3.11/site-packages/urllib3/util/ssl_.py)'].
monkey.patch_all(thread=False)
Please execute command `export MINDIE_LOG_TO_STDOUT="benchmark:1; client:1"` if log message need to be displayed on screen next time.
[2025-03-19 18:06:28.424+08:00] [25173] [281473893255904] [benchmark] [INFO] [benchmark_run.py:93] Running Task.
[2025-03-19 18:06:28.425+08:00] [25173] [281473893255904] [benchmark] [INFO] [benchmark_run.py:103] [UserId]: root, [Ip]: localhost, [Model]: deepseek-r1, [TaskType]: Performance, [TestType]: client, [DatasetType]: gsm8k, [Http]: http
://172.20.112.20:1205
[2025-03-19 18:06:28.425+08:00] [25173] [281473893255904] [benchmark] [INFO] [benchmark_run.py:46] The Benchmark init instance_path: ./instance
Benchmark task is running, please wait...
[2025-03-19 18:06:28.884+08:00] [25173] [281462532665760] [benchmark] [INFO] [base_loader.py:103] Reading dataset...
[2025-03-19 18:06:28.885+08:00] [25173] [281462532665760] [benchmark] [INFO] [base_loader.py:112] Finished loading dataset...
[2025-03-19 18:06:28.885+08:00] [25173] [281473893255904] [benchmark] [INFO] [benchmarker.py:72] MULTI MACHINE is running
[2025-03-19 18:06:28.886+08:00] [25173] [281473893255904] [benchmark] [INFO] [benchmarker.py:76] Multi machine is running in [slave] machine node
[2025-03-19 18:06:28.886+08:00] [25173] [281473893255904] [benchmark] [INFO] [benchmarker.py:44] Starting benchmark...
[2025-03-19 18:06:28.886+08:00] [25173] [281473893255904] [benchmark] [INFO] [benchmarker.py:122] Starting run mindie-client inference
[2025-03-19 18:06:28.888+08:00] [25173] [281473893255904] [client] [WARN] [base_client.py:145] [MIE02W000007] SSL authentication is disabled!
[2025-03-19 18:06:28.888+08:00] [25173] [281473893255904] [benchmark] [INFO] [base_api.py:77] DoSampling is disabled
[2025-03-19 18:06:28.889+08:00] [25173] [281473893255904] [benchmark] [INFO] [base_api.py:82] Sampling params is {}.
[2025-03-19 18:06:28.889+08:00] [25173] [281473893255904] [benchmark] [INFO] [base_api.py:89] Client init finished. Gevent pool size is 128, max connection for mindie-client is 1000.
[2025-03-19 18:06:28.889+08:00] [25173] [281473893255904] [benchmark] [INFO] [base_api.py:164] Reset states...
[2025-03-19 18:06:28.889+08:00] [25173] [281473893255904] [benchmark] [WARN] [base_api.py:173] [MIE01W000102] Do not check the status of server because management port is disabled
[2025-03-19 18:06:28.898+08:00] [25173] [281473893255904] [client] [ERROR] [utils.py:47] Request failed and the reason is HTTPConnectionPool(host='172.20.112.20', port=1206): Max retries exceeded with url: /v2/health/ready (Caused by
NewConnectionError(': Failed to establish a new connection: [Errno 111] Connection refused')).
[2025-03-19 18:06:28.898+08:00] [25173] [281473893255904] [client] [WARN] [base_client.py:145] [MIE02W000007] SSL authentication is disabled!
[2025-03-19 18:06:28.898+08:00] [25173] [281473893255904] [benchmark] [INFO] [base_api.py:77] DoSampling is disabled
[2025-03-19 18:06:28.898+08:00] [25173] [281473893255904] [benchmark] [INFO] [base_api.py:82] Sampling params is {}.
[2025-03-19 18:06:28.899+08:00] [25173] [281473893255904] [benchmark] [INFO] [base_api.py:89] Client init finished. Gevent pool size is 128, max connection for mindie-client is 1000.
[2025-03-19 18:06:28.899+08:00] [25173] [281473893255904] [benchmark] [INFO] [base_api.py:164] Reset states...
[2025-03-19 18:06:28.899+08:00] [25173] [281473893255904] [benchmark] [WARN] [base_api.py:173] [MIE01W000102] Do not check the status of server because management port is disabled
[2025-03-19 18:06:28.899+08:00] [25173] [281473893255904] [benchmark] [INFO] [base_api.py:318] Warm up start...
[2025-03-19 18:06:28.914+08:00] [25173] [281473893255904] [client] [ERROR] [utils.py:47] [MIE02E000002] The prompt should be a string or a list!
[2025-03-19 18:06:28.915+08:00] [25173] [281473893255904] [benchmark] [ERROR] [base_api.py:348] [MIE01E000108] Warm up request failed, request warm_up_0_1cc83c69593647efb4c589c32f5e55d3 occurs error with message [MIE02E000002] The pro
mpt should be a string or a list!.
[2025-03-19 18:06:28.915+08:00] [25173] [281473893255904] [client] [ERROR] [utils.py:47] [MIE02E000002] The prompt should be a string or a list!
[2025-03-19 18:06:28.915+08:00] [25173] [281473893255904] [benchmark] [ERROR] [base_api.py:348] [MIE01E000108] Warm up request failed, request warm_up_1_ba7300defe7a4ada8a13ea8ac33aa586 occurs error with message [MIE02E000002] The pro
mpt should be a string or a list!.
[2025-03-19 18:06:28.915+08:00] [25173] [281473893255904] [client] [ERROR] [utils.py:47] [MIE02E000002] The prompt should be a string or a list!
[2025-03-19 18:06:28.916+08:00] [25173] [281473893255904] [benchmark] [ERROR] [base_api.py:348] [MIE01E000108] Warm up request failed, request warm_up_2_cc83b004e6d040f5b3d20ccf383a7c25 occurs error with message [MIE02E000002] The pro
mpt should be a string or a list!.
[2025-03-19 18:06:28.916+08:00] [25173] [281473893255904] [client] [ERROR] [utils.py:47] [MIE02E000002] The prompt should be a string or a list!
[2025-03-19 18:06:28.916+08:00] [25173] [281473893255904] [benchmark] [ERROR] [base_api.py:348] [MIE01E000108] Warm up request failed, request warm_up_3_8fa19475bb6745bbb6652b5d9a355d3d occurs error with message [MIE02E000002] The pro
mpt should be a string or a list!.
[2025-03-19 18:06:28.916+08:00] [25173] [281473893255904] [client] [ERROR] [utils.py:47] [MIE02E000002] The prompt should be a string or a list!
[2025-03-19 18:06:28.917+08:00] [25173] [281473893255904] [benchmark] [ERROR] [base_api.py:348] [MIE01E000108] Warm up request failed, request warm_up_4_119ba95738e243c7bd0c7149bb1de2c4 occurs error with message [MIE02E000002] The pro
mpt should be a string or a list!.
[2025-03-19 18:06:28.917+08:00] [25173] [281473893255904] [client] [ERROR] [utils.py:47] [MIE02E000002] The prompt should be a string or a list!
[2025-03-19 18:06:28.917+08:00] [25173] [281473893255904] [benchmark] [ERROR] [base_api.py:348] [MIE01E000108] Warm up request failed, request warm_up_5_bc8b2ee814bf4c45ab11d861fc268999 occurs error with message [MIE02E000002] The pro
mpt should be a string or a list!.
[2025-03-19 18:06:28.917+08:00] [25173] [281473893255904] [client] [ERROR] [utils.py:47] [MIE02E000002] The prompt should be a string or a list!
[2025-03-19 18:06:28.918+08:00] [25173] [281473893255904] [benchmark] [ERROR] [base_api.py:348] [MIE01E000108] Warm up request failed, request warm_up_6_ca4a3523da294b09b9a11105408b5cc3 occurs error with message [MIE02E000002] The pro
mpt should be a string or a list!.
[2025-03-19 18:06:28.918+08:00] [25173] [281473893255904] [client] [ERROR] [utils.py:47] [MIE02E000002] The prompt should be a string or a list!
[2025-03-19 18:06:28.918+08:00] [25173] [281473893255904] [benchmark] [ERROR] [base_api.py:348] [MIE01E000108] Warm up request failed, request warm_up_7_fd060518395c40af91f641ff4ef99203 occurs error with message [MIE02E000002] The pro
mpt should be a string or a list!.
[2025-03-19 18:06:28.918+08:00] [25173] [281473893255904] [client] [ERROR] [utils.py:47] [MIE02E000002] The prompt should be a string or a list!
[2025-03-19 18:06:28.919+08:00] [25173] [281473893255904] [benchmark] [ERROR] [base_api.py:348] [MIE01E000108] Warm up request failed, request warm_up_8_a6ea1226913a48bfb1f5e4ad7c718470 occurs error with message [MIE02E000002] The pro
mpt should be a string or a list!.
[2025-03-19 18:06:28.919+08:00] [25173] [281473893255904] [client] [ERROR] [utils.py:47] [MIE02E000002] The prompt should be a string or a list!
[2025-03-19 18:06:28.919+08:00] [25173] [281473893255904] [benchmark] [ERROR] [base_api.py:348] [MIE01E000108] Warm up request failed, request warm_up_9_5ce0664edea349c3ad838be4ef494e90 occurs error with message [MIE02E000002] The pro
mpt should be a string or a list!.
[2025-03-19 18:06:28.919+08:00] [25173] [281473893255904] [benchmark] [INFO] [base_api.py:335] Warm up finished. Handling 10 requests, cost warm up time 0.006 seconds
[2025-03-19 18:06:28.920+08:00] [25173] [281473893255904] [benchmark] [INFO] [base_api.py:295] Handling 0 requests, failed 0
[2025-03-19 18:06:28.920+08:00] [25173] [281473893255904] [benchmark] [ERROR] [base_api.py:237] [MIE01E000005] All requests failed, please check dataset path or other settings.
[2025-03-19 18:06:28.920+08:00] [25173] [281473893255904] [benchmark] [INFO] [output.py:280] calc_common_metrics start time: 1742378788.8863451 !
[2025-03-19 18:06:28.921+08:00] [25173] [281473893255904] [benchmark] [WARN] [output.py:282] [MIE01W000005] Empty response numbers: 0 !
[2025-03-19 18:06:28.924+08:00] [25173] [281473893255904] [benchmark] [INFO] [output.py:115]
The BenchMark test performance metric result is:
+---------------------+-----------+-----------+-----------+-----------+-----------+-----------+-----------+---+
| Metric | average | max | min | P75 | P90 | SLO_P90 | P99 | N |
+---------------------+-----------+-----------+-----------+-----------+-----------+-----------+-----------+---+
| FirstTokenTime | 0 ms | 0 ms | 0 ms | 0 ms | 0 ms | 0 ms | 0 ms | 0 |
| DecodeTime | 0 ms | 0 ms | 0 ms | 0 ms | 0 ms | 0 ms | 0 ms | 0 |
| LastDecodeTime | 0 ms | 0 ms | 0 ms | 0 ms | 0 ms | 0 ms | 0 ms | 0 |
| MaxDecodeTime | 0 ms | 0 ms | 0 ms | 0 ms | 0 ms | 0 ms | 0 ms | 0 |
| GenerateTime | 0 ms | 0 ms | 0 ms | 0 ms | 0 ms | 0 ms | 0 ms | 0 |
| InputTokens | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 |
| GeneratedTokens | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 |
| GeneratedTokenSpeed | 0 token/s | 0 token/s | 0 token/s | 0 token/s | 0 token/s | 0 token/s | 0 token/s | 0 |
| GeneratedCharacters | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 |
| Tokenizer | 0 ms | 0 ms | 0 ms | 0 ms | 0 ms | 0 ms | 0 ms | 0 |
| Detokenizer | 0 ms | 0 ms | 0 ms | 0 ms | 0 ms | 0 ms | 0 ms | 0 |
| CharactersPerToken | 0 | / | / | / | / | / | / | 0 |
| PostProcessingTime | 0 ms | 0 ms | 0 ms | 0 ms | 0 ms | 0 ms | 0 ms | 0 |
| ForwardTime | 0 ms | 0 ms | 0 ms | 0 ms | 0 ms | 0 ms | 0 ms | 0 |
+---------------------+-----------+-----------+-----------+-----------+-----------+-----------+-----------+---+
[2025-03-19 18:06:28.926+08:00] [25173] [281473893255904] [benchmark] [INFO] [output.py:121]
The BenchMark test common metric result is:
+------------------------+---------------------+
| Common Metric | Value |
+------------------------+---------------------+
| CurrentTime | 2025-03-19 18:06:28 |
| TimeElapsed | 0 s |
| DataSource | /root/gsm8k/ |
| Failed | 0( 0 ) |
| Returned | 0( 0 ) |
| Total | 0[ 0 ] |
| Concurrency | 128 |
| ModelName | deepseek-r1 |
| lpct | 0 ms |
| Throughput | 0 req/s |
| GenerateSpeedPerClient | 0 token/s |
| accuracy | / |
+------------------------+---------------------+
[2025-03-19 18:06:28.926+08:00] [25173] [281473893255904] [benchmark] [WARN] [visual_utils.py:130] [MIE01W000004] No data to draw histogram: Frequency_Histogram_of_Decoder_Batch_Size_None.png, return. Please verify which mode has the
data to plot
[2025-03-19 18:06:28.926+08:00] [25173] [281473893255904] [benchmark] [WARN] [visual_utils.py:130] [MIE01W000004] No data to draw histogram: Frequency_Histogram_of_Prefill_Batch_Size_None.png, return. Please verify which mode has the
data to plot
[2025-03-19 18:06:28.927+08:00] [25173] [281473893255904] [benchmark] [WARN] [visual_utils.py:130] [MIE01W000004] No data to draw histogram: Frequency_Histogram_of_Decode_Token_Latencies_None.png, return. Please verify which mode has
the data to plot
[2025-03-19 18:06:28.927+08:00] [25173] [281473893255904] [benchmark] [WARN] [visual_utils.py:130] [MIE01W000004] No data to draw histogram: Frequency_Histogram_of_Prefill_Token_Latencies_None.png, return. Please verify which mode has
the data to plot
[2025-03-19 18:06:28.927+08:00] [25173] [281473893255904] [benchmark] [WARN] [visual_utils.py:130] [MIE01W000004] No data to draw histogram: Frequency_Histogram_of_Queue_Wait_Time_None.png, return. Please verify which mode has the dat
a to plot
[2025-03-19 18:06:28.927+08:00] [25173] [281473893255904] [benchmark] [WARN] [visual_utils.py:130] [MIE01W000004] No data to draw histogram: Frequency_Histogram_of_Tokenizer_Time_None.png, return. Please verify which mode has the data
to plot
[2025-03-19 18:06:28.928+08:00] [25173] [281473893255904] [benchmark] [WARN] [visual_utils.py:130] [MIE01W000004] No data to draw histogram: Frequency_Histogram_of_Detokenizer_Time_None.png, return. Please verify which mode has the da
ta to plot
[2025-03-19 18:06:28.928+08:00] [25173] [281473893255904] [benchmark] [WARN] [visual_utils.py:130] [MIE01W000004] No data to draw histogram: Frequency_Histogram_of_Post_Processing_Latencies_None.png, return. Please verify which mode h
as the data to plot
[2025-03-19 18:06:28.928+08:00] [25173] [281473893255904] [benchmark] [WARN] [visual_utils.py:178] [MIE01W000004] No data to plot: Scatter_Plot_of_Decoder_Batch_Size_vs_Decode_Token_Latencies_None.png, return.Please verify which mode
has the data to plot
[2025-03-19 18:06:28.928+08:00] [25173] [281473893255904] [benchmark] [WARN] [visual_utils.py:178] [MIE01W000004] No data to plot: Scatter_Plot_of_Prefill_Sequence_Length_vs_Prefill_Token_Latencies_None.png, return.Please verify which
mode has the data to plot
[2025-03-19 18:06:28.929+08:00] [25173] [281473893255904] [benchmark] [WARN] [visual_utils.py:178] [MIE01W000004] No data to plot: Scatter_Plot_of_Prefill_Sequence_Length_vs_Prefill_Token_Per_Second_None.png, return.Please verify whic
h mode has the data to plot
Benchmark task completed successfully!