按照案例里面的百川模型推理报错
收藏回复举报
按照案例里面的百川模型推理报错
t('forum.solved') 已解决
发表于2023-12-28 16:33:53
0 查看
(chatglm_py39) [sy_cmsr@zq01-wtd-test-AIGC-0002 Baichuan2-13b]$ bash /home/sy_cmsr/work/Baichuan2-13b/run_singlenode.sh "python3.9 /home/sy_cmsr/work/Baichuan2-13b/run_baichuan2_generate.py" /home/sy_cmsr/work/Baichuan2-13b/hccl_8p_01234567_10.125.80.11.json [0,8] 8 

start training for rank 0, device 0 

start training for rank 1, device 1 

start training for rank 2, device 2 

start training for rank 3, device 3 

start training for rank 4, device 4 

start training for rank 5, device 5 

start training for rank 6, device 6 

start training for rank 7, device 7

用上述命令分8张卡跑,配置文件也照着案例写的,生成了output文件夹,为什么会报下面这个错误呢,transformed_checkpoint里面也没有东西,推理也没有结果?

Traceback (most recent call last):

  File "/home/sy_cmsr/work/Baichuan2-13b/run_baichuan2_generate.py", line 64, in <module>

    transform_and_load_checkpoint(baichuan2_config, baichuan2_model, baichuan2_network, infer_data, do_predict=True)

  File "/home/sy_cmsr/miniconda3/envs/chatglm_py39/lib/python3.9/site-packages/mindformers/trainer/utils.py", line 328, in transform_and_load_checkpoint

    transform_ckpt(config, src_ckpt_strategy=src_ckpt_strategy, dst_ckpt_strategy=dst_ckpt_strategy)

  File "/home/sy_cmsr/miniconda3/envs/chatglm_py39/lib/python3.9/site-packages/mindformers/trainer/utils.py", line 460, in transform_ckpt

    ms.transform_checkpoints(config.load_checkpoint,

  File "/home/sy_cmsr/miniconda3/envs/chatglm_py39/lib/python3.9/site-packages/mindspore/parallel/checkpoint_transform.py", line 302, in transform_checkpoints

    ckpt_dict = ms.load_checkpoint(all_checkpoint_files_map.get(int(needed_rank)))

  File "/home/sy_cmsr/miniconda3/envs/chatglm_py39/lib/python3.9/site-packages/mindspore/train/serialization.py", line 1097, in load_checkpoint

    raise ValueError(f"The loaded parameter dict is empty after filter or specify, please check whether "

ValueError: The loaded parameter dict is empty after filter or specify, please check whether 'filter_prefix' or 'specify_prefix' are set correctly.

我要发帖子