DeepSeek-R1-Distill-Llama-8B在MindIE上做MMLU数据集精度测试时 Invalid response format, please check the message in mindie-client.
收藏回复举报
DeepSeek-R1-Distill-Llama-8B在MindIE上做MMLU数据集精度测试时 Invalid response format, please check the message in mindie-client.
t('forum.solved') 已解决
发表于2025-02-26 10:45:53
0 查看

执行命令:

(Python310) root@bms-41ba-0002:/usr/local/Ascend/mindie/latest/mindie-service# benchmark --DatasetPath "/root/model_pt/data/mmlu/" --DatasetType "mmlu" --ModelName llama --ModelPath "/root/model_pt/DeepSeek-R1-Distill-Llama-8B/" --TestType client --Http http://127.0.0.1:1025 --ManagementHttp http://127.0.0.2:1026 --Concurrency 256 --TaskKind stream --Tokenizer True --MaxOutputLen 512 --TestAccuracy True

报错:

性能测试方报错,精度也很低

cke_2663.png

服务化方报错

cke_6758.png

我要发帖子