按照https://gitcode.com/Ascend/MindSpeed-LLM/blob/master/docs/zh/pytorch/training/finetune/mcore/layerwise_disaggregated_training.md指导文档部署边云协同分布式安全训练。
在examples/mcore/qwen3/ckpt_convert_qwen3_hf2mcore.sh脚本中,添加--num-layer-list参数后,执行脚本会上报如下错误:
Traceback (most recent call last):
File "/MindSpeed-LLM/MindSpeed-LLM/convert_ckpt_v2.py", line 104, in <module>
main()
File "/MindSpeed-LLM/MindSpeed-LLM/mindspeed_llm/training/utils.py", line 1106, in wrapper
return func(*args, **kwargs)
File "/MindSpeed-LLM/MindSpeed-LLM/convert_ckpt_v2.py", line 91, in main
converter = Hf2MgConvert(args)
File "/MindSpeed-LLM/MindSpeed-LLM/mindspeed_llm/tasks/checkpoint/convert_hf2mg.py", line 76, in __init__
self.get_pprank_hf_layeridxs()
File "/MindSpeed-LLM/MindSpeed-LLM/mindspeed_llm/tasks/checkpoint/convert_hf2mg.py", line 164, in get_pprank_hf_layeridxs
self.pprank_layer_idxs[pp_rank] = [num_layer_list_.pop(0) for _ in range(layers_each_pp[pp_rank])]
File "/MindSpeed-LLM/MindSpeed-LLM/mindspeed_llm/tasks/checkpoint/convert_hf2mg.py", line 164, in <listcomp>
self.pprank_layer_idxs[pp_rank] = [num_layer_list_.pop(0) for _ in range(layers_each_pp[pp_rank])]
IndexError: pop from empty list
[ERROR] 2026-04-10-16:39:19 (PID:16057, Device:-1, RankID:-1) ERR99999 UNKNOWN applicaiton exception
删除--num-layer-list 16,32,16 # U-shape切分后,权重转换脚本可正常运行
按照https://gitcode.com/Ascend/MindSpeed-LLM/blob/master/docs/zh/pytorch/training/finetune/mcore/layerwise_disaggregated_training.md指导文档部署边云协同分布式安全训练。
在examples/mcore/qwen3/ckpt_convert_qwen3_hf2mcore.sh脚本中,添加--num-layer-list参数后,执行脚本会上报如下错误:
Traceback (most recent call last):
File "/MindSpeed-LLM/MindSpeed-LLM/convert_ckpt_v2.py", line 104, in <module>
main()
File "/MindSpeed-LLM/MindSpeed-LLM/mindspeed_llm/training/utils.py", line 1106, in wrapper
return func(*args, **kwargs)
File "/MindSpeed-LLM/MindSpeed-LLM/convert_ckpt_v2.py", line 91, in main
converter = Hf2MgConvert(args)
File "/MindSpeed-LLM/MindSpeed-LLM/mindspeed_llm/tasks/checkpoint/convert_hf2mg.py", line 76, in __init__
self.get_pprank_hf_layeridxs()
File "/MindSpeed-LLM/MindSpeed-LLM/mindspeed_llm/tasks/checkpoint/convert_hf2mg.py", line 164, in get_pprank_hf_layeridxs
self.pprank_layer_idxs[pp_rank] = [num_layer_list_.pop(0) for _ in range(layers_each_pp[pp_rank])]
File "/MindSpeed-LLM/MindSpeed-LLM/mindspeed_llm/tasks/checkpoint/convert_hf2mg.py", line 164, in <listcomp>
self.pprank_layer_idxs[pp_rank] = [num_layer_list_.pop(0) for _ in range(layers_each_pp[pp_rank])]
IndexError: pop from empty list
[ERROR] 2026-04-10-16:39:19 (PID:16057, Device:-1, RankID:-1) ERR99999 UNKNOWN applicaiton exception
删除--num-layer-list 16,32,16 # U-shape切分后,权重转换脚本可正常运行