deepseek-r1做w4a8权重量化,推理报错AttributeError: 'DeepseekV2RMSNorm' object has no attribute 'ori'
收藏回复举报
deepseek-r1做w4a8权重量化,推理报错AttributeError: 'DeepseekV2RMSNorm' object has no attribute 'ori'
t('forum.solved') 已解决
发表于2025-05-15 14:54:26
0 查看

使用mindie-2.0.RC1镜像,设备是800IA2(64G);

问题描述deepseek-r1权重w4a8量化后,mindie-service起推理服务,报错AttributeError: 'DeepseekV2RMSNorm' object has no attribute 'ori'

权重量化操作参考如下截图:
cke_2245.png

权重量化完成截图如下:
cke_5534.png

mindie的config.json修改如下,其余保持默认:

"httpsEnabled"=false

"npuDeviceIds"=[[0,1,2,3,4,5,6,7]]

"modelName"=deepseek-r1-w4a8

"modelWeightPath"=${权重实际路径}

"worldSize"=8

"maxSeqLen"=32768

"maxInputTokenLen"=16384

"maxPrefillTokens"=16384

"maxIterTimes"=16384

推理报错截图:

cke_64149.png

cke_65479.png

cke_67511.png

我要发帖子