1 系统环境
硬件环境(Ascend/GPU/CPU): Ascend
MindSpore版本: mindspore=2.4.1、mindformers==1.3.0
执行模式(PyNative/ Graph):不限
Python版本: Python=3.9
操作系统平台: linux
2 报错信息
2.1 问题描述
参考msrun启动教程:不设置device_target会自动指定为MindSpore包对应的后端硬件设备,但是通过以下命令发现识别到的NPU卡数量为1,导致MSRUN跑并行任务时只有一个NPU节点有日志。
2.2 脚本信息
2.3 报错信息
3 根因分析
******此处由用户填写******
4 解决方案
******此处由用户填写******
包含文字方案和最终脚本代码
请将正确的脚本打包并上传附件
1 系统环境
硬件环境(Ascend/GPU/CPU): Ascend
MindSpore版本: mindspore=2.4.1、mindformers==1.3.0
执行模式(PyNative/ Graph):不限
Python版本: Python=3.9
操作系统平台: linux
2 报错信息
2.1 问题描述
参考msrun启动教程:不设置device_target会自动指定为MindSpore包对应的后端硬件设备,但是通过以下命令发现识别到的NPU卡数量为1,导致MSRUN跑并行任务时只有一个NPU节点有日志。
2.2 脚本信息
import mindspore as ms ms.set_context(mode=ms.GRAPH_MODE, device_target="Ascend") device_num = ms.context.get_auto_parallel_context("device_num") print("MindSpore识别到的NPU卡数量为:", device_num)2.3 报错信息
(MindSpore) [root ringmo] $npu-smi info +------------------------------------------------------------------------------------------------------------------------------+ | npu-smi 23.0.rc2.0 Version: 24.1.rc2.1 | +------------------------+------------------------+---------------------------------------------------------------------------+ | NPU Name | Health | Power(W) Temp(C) Hugepages-Usage(page) | | Chip | Bus-Id | AICore(%) Memory-Usage(MB) HBM-Usage(MB) | +==============+==============+============================================+ | 3 910 | OK | 97.9 45 0 / 0 | | 0 | 0000:02:00.0 | 0 0 / 0 3673 / 65536 | +==============+==============+============================================+ | 5 910 | OK | 94.5 45 0 / 0 | | 0 | 0000:41:00.0 | 0 0 / 0 3340 / 65536 | +==============+==============+============================================+ +------------------------+------------------------+---------------------------------------------------------------------------+ | NPU Chip | Process id | Process name | Process memory(MB) | +==============+==============+============================================+ | No running process found in NPU 3 | +==============+==============+============================================+ | No running process found in NPU 5 | +==============+==============+============================================+ (MindSpore) [root ringmo] $python Python 3.9.18 | packaged by conda-forge | (main, Aug 30 2023, 04: 25: 25) [GCC 12.3.0] on Linux Type "help", "copyright", "credits" or "license" for more information. >>> inport nindspore as ms >>> >>> ms.set_ context(mode=ns.GRAPH_MODE, device_target="Ascend") >>> device_num = ms.context.get_auto_parallel_context("device_num") >>> print("MindSpore识别到的NPU卡数量为:", device_num) MindSpore识别到的NPU卡数量为: 13 根因分析
******此处由用户填写******
4 解决方案
******此处由用户填写******
包含文字方案和最终脚本代码
请将正确的脚本打包并上传附件