使用mindie-2.0.RC1镜像,设备是800IA2(64G);
问题描述:deepseek-r1权重w4a8量化后,mindie-service起推理服务,报错AttributeError: 'DeepseekV2RMSNorm' object has no attribute 'ori'
权重量化操作参考如下截图:

权重量化完成截图如下:

mindie的config.json修改如下,其余保持默认:
"httpsEnabled"=false
"npuDeviceIds"=[[0,1,2,3,4,5,6,7]]
"modelName"=deepseek-r1-w4a8
"modelWeightPath"=${权重实际路径}
"worldSize"=8
"maxSeqLen"=32768
"maxInputTokenLen"=16384
"maxPrefillTokens"=16384
"maxIterTimes"=16384
推理报错截图:



使用mindie-2.0.RC1镜像,设备是800IA2(64G);
问题描述:deepseek-r1权重w4a8量化后,mindie-service起推理服务,报错AttributeError: 'DeepseekV2RMSNorm' object has no attribute 'ori'
权重量化操作参考如下截图:

权重量化完成截图如下:

mindie的config.json修改如下,其余保持默认:
"httpsEnabled"=false
"npuDeviceIds"=[[0,1,2,3,4,5,6,7]]
"modelName"=deepseek-r1-w4a8
"modelWeightPath"=${权重实际路径}
"worldSize"=8
"maxSeqLen"=32768
"maxInputTokenLen"=16384
"maxPrefillTokens"=16384
"maxIterTimes"=16384
推理报错截图: