一张910B，利用mindspeed训练qwen3-8b模型，出现了显存不足的情况，有什么办法设置下参数，可以达到训练的目的，报错信息如下：
RuntimeError: NPU out of memory. Tried to allocate 30.51 GiB (NPU 0; 60.97 GiB total capacity; 30.51 GiB already allocated; 30.51 GiB current active; 30.11 GiB free; 30.53 GiB reserved in total by PyTorch) If reserved memory is >> allocated memory try setting max_split_size_mb to avoid fragmentation.
[ERROR] 2025-08-13-09:16:01 (PID:3387, Device:0, RankID:-1) ERR99999 UNKNOWN application exception
点赞
收藏
回复
分享
举报
已解决
一张910B，利用mindspeed训练qwen3-8b模型，出现了显存不足的情况，有什么办法设置下参数，可以达到训练的目的，报错信息如下：
RuntimeError: NPU out of memory. Tried to allocate 30.51 GiB (NPU 0; 60.97 GiB total capacity; 30.51 GiB already allocated; 30.51 GiB current active; 30.11 GiB free; 30.53 GiB reserved in total by PyTorch) If reserved memory is >> allocated memory try setting max_split_size_mb to avoid fragmentation.
[ERROR] 2025-08-13-09:16:01 (PID:3387, Device:0, RankID:-1) ERR99999 UNKNOWN application exception