麻烦您帮忙看看,我在训练图像描述生成模型时,输入数据集是ms coco 2017,然后在训练过程发生报错,调试时打断点也找不到报错代码位置
收藏回复举报
麻烦您帮忙看看,我在训练图像描述生成模型时,输入数据集是ms coco 2017,然后在训练过程发生报错,调试时打断点也找不到报错代码位置
t('forum.solved') 已解决
发表于2023-05-05 16:14:15
0 查看

麻烦您帮忙看看,我在训练图像描述生成模型时,输入数据集是ms coco 2017,然后在训练过程发生报错,调试时打断点到loss, loss_info = self.model(self.rl_stage, indices, input_seq, target_seq, gv_feat, att_feats, att_mask)这句话然后逐步调试会直接跳出,没有进入到construct函数中... 所以找不到报错位置

for epoch in range(self.load_epoch + 1, cfg.SOLVER.MAX_EPOCH):
            # 每一个Epoch内部Iteration迭代
            with tqdm.tqdm(desc='Epoch %d - train' % epoch, unit='it', total=self.training_loader.get_dataset_size()) as pbar:
                for data in self.training_loader.create_dict_iterator():
                    print("---------------------Epoch内部Iteration迭代---------------------")
                    print(data)
                    indices, input_seq, target_seq, gv_feat, att_feats, att_mask = self.make_kwargs(data["indices"], data["input_seq"], data["target_seq"], data["gv_feat"], data["att_feats"], data["att_mask"])
                    loss, loss_info = self.model(self.rl_stage, indices, input_seq, target_seq, gv_feat, att_feats, att_mask)

其中data可以正常输出:

{'indices': Tensor(shape=[36], dtype=Int64, value= [ 42364,  12825,  55881,  68550,  ...  37732]), 
  'input_seq': Tensor(shape=[36, 5, 17], dtype=Int64, value=
[[[   0, 
...   1,    0]]]),
  'target_seq': Tensor(shape=[36, 5, 17], dtype=Int64, value=
[[[2500, 4019, ... -1,   -1]]]), 
  'gv_feat': Tensor(shape=[36, 1], dtype=Float64, value=
[[ 0.00000000e+00],...[ 0.00000000e+00]]), 
 'att_feats': Tensor(shape=[36, 144, 1536], dtype=Float32, value=
[[[ 1.78379081e-02,  ... -9.93363326e-04]]]), 
'att_mask': Tensor(shape=[36, 144], dtype=Float64, value=
[[ 1.00000000e+00,  ...  1.00000000e+00]])}

但是运行报错:

[CRITICAL] ME(606029:281473551325760,MainProcess):2023-05-05-16:05:30.765.960 [mindspore/dataset/engine/datasets.py:2778] Uncaught exception: 
Traceback (most recent call last):
  File "/home/ma-user/work/PureT/main.py", line 416, in <module>
    trainer.train()
  File "/home/ma-user/work/PureT/main.py", line 350, in train
    loss, loss_info = self.model(self.rl_stage, indices, input_seq, target_seq, gv_feat, att_feats, att_mask)
  File "/home/ma-user/anaconda3/envs/MindSpore/lib/python3.7/site-packages/mindspore/nn/cell.py", line 586, in __call__
    out = self.compile_and_run(*args)
  File "/home/ma-user/anaconda3/envs/MindSpore/lib/python3.7/site-packages/mindspore/nn/cell.py", line 964, in compile_and_run
    self.compile(*inputs)
  File "/home/ma-user/anaconda3/envs/MindSpore/lib/python3.7/site-packages/mindspore/nn/cell.py", line 937, in compile
    _cell_graph_executor.compile(self, *inputs, phase=self.phase, auto_parallel_mode=self._auto_parallel_mode)
  File "/home/ma-user/anaconda3/envs/MindSpore/lib/python3.7/site-packages/mindspore/common/api.py", line 1006, in compile
    result = self._graph_executor.compile(obj, args_list, phase, self._use_vm_mode())
TypeError: 'int' object is not iterable

代码中的this.model如下:

class PureTTrainOneStepCell(nn.TrainOneStepCell):
    def __init__(self, network, optimizer, sens=1.0):
        super(PureTTrainOneStepCell, self).__init__(network, optimizer, sens)
        self.cast = ops.Cast()
        self.hyper_map = ops.HyperMap()

    def set_sens(self, value):
        self.sens = value
    
    def construct(self, rl_stage, indices, input_seq, target_seq, gv_feat, att_feats, att_mask):
        print(rl_stage)
        print('PureTTrainOneStepCell\n')
        weights = self.weights
        loss, loss_info = self.network(rl_stage, indices, input_seq, target_seq, gv_feat, att_feats, att_mask)
        grads = self.grad(self.network, weights)(rl_stage, indices, input_seq, target_seq, gv_feat, att_feats, att_mask, self.cast(ops.tuple_to_array((self.sens,)), ms.float32))
        grads = self.hyper_map(ops.partial(utils.clip_grad, cfg.SOLVER.GRAD_CLIP_TYPE, cfg.SOLVER.GRAD_CLIP), grads)
        grads = self.grad_reducer(grads)
        self.optimizer(grads)
        return loss, loss_info

我要发帖子