deepseek显存占用情况
收藏回复举报
deepseek显存占用情况
t('forum.solved') 已解决
发表于2025-02-18 14:22:40
0 查看
请问有优化deepseek模型在mindie的显存占用情况吗?我的启动方式,显存占用太大了,感谢建议指正🙏。
model显存
deepseek-7b-w8a856g-2卡
deepseek-7b56g-2卡
deepseek-14b-w8a899g-4卡
deepseek-14b105g-4卡

启动模型方式: 

修改/usr/local/Ascend/mindie/latest/mindie-service/conf/config.json文件参数 

data["ServerConfig"]["port"] = mindie_port 

data["ServerConfig"]["httpsEnabled"] = False 

data["ServerConfig"]["fullTextEnabled"] = True 

data["BackendConfig"]["npuDeviceIds"] = [gpus] 

data["BackendConfig"]["ModelDeployConfig"]["maxSeqLen"] = max_seq_len * 2 

data["BackendConfig"]["ModelDeployConfig"]["maxInputTokenLen"] = max_seq_len 

data["BackendConfig"]["ModelDeployConfig"]["ModelConfig"][0][ "modelWeightPath" ] = config.model_path data["BackendConfig"]["ModelDeployConfig"]["ModelConfig"][0][ "modelName" ] = ",".join( self.model_names, ) data["BackendConfig"]["ModelDeployConfig"]["ModelConfig"][0][ "worldSize" ] = len(gpus) 

data["BackendConfig"]["ModelDeployConfig"]["ModelConfig"][0][ "cpuMemSize" ] = cpu_mem_size data["BackendConfig"]["ModelDeployConfig"]["ModelConfig"][0][ "npuMemSize" ] = npu_mem_size data["BackendConfig"]["ScheduleConfig"]["maxPrefillTokens"] = ( max_seq_len * 2 ) # 输出tokens data["BackendConfig"]["ScheduleConfig"]["maxIterTimes"] = max_seq_len 


 再启动服务:/usr/local/Ascend/mindie/latest/mindie-service/bin/mindieservice_daemon 

process = subprocess.Popen( 

    ["/usr/local/Ascend/mindie/latest/mindie-service/bin/mindieservice_daemon"], 

    stdout=subprocess.PIPE, 

    stderr=subprocess.STDOUT, 

我要发帖子