麻烦您帮忙看看,我在训练图像描述生成模型时,输入数据集是ms coco 2017,然后在训练过程发生报错,调试时打断点到loss, loss_info = self.model(self.rl_stage, indices, input_seq, target_seq, gv_feat, att_feats, att_mask)这句话然后逐步调试会直接跳出,没有进入到construct函数中... 所以找不到报错位置
其中data可以正常输出:
但是运行报错:
代码中的this.model如下:
麻烦您帮忙看看,我在训练图像描述生成模型时,输入数据集是ms coco 2017,然后在训练过程发生报错,调试时打断点到loss, loss_info = self.model(self.rl_stage, indices, input_seq, target_seq, gv_feat, att_feats, att_mask)这句话然后逐步调试会直接跳出,没有进入到construct函数中... 所以找不到报错位置
for epoch in range(self.load_epoch + 1, cfg.SOLVER.MAX_EPOCH): # 每一个Epoch内部Iteration迭代 with tqdm.tqdm(desc='Epoch %d - train' % epoch, unit='it', total=self.training_loader.get_dataset_size()) as pbar: for data in self.training_loader.create_dict_iterator(): print("---------------------Epoch内部Iteration迭代---------------------") print(data) indices, input_seq, target_seq, gv_feat, att_feats, att_mask = self.make_kwargs(data["indices"], data["input_seq"], data["target_seq"], data["gv_feat"], data["att_feats"], data["att_mask"]) loss, loss_info = self.model(self.rl_stage, indices, input_seq, target_seq, gv_feat, att_feats, att_mask)其中data可以正常输出:
{'indices': Tensor(shape=[36], dtype=Int64, value= [ 42364, 12825, 55881, 68550, ... 37732]), 'input_seq': Tensor(shape=[36, 5, 17], dtype=Int64, value= [[[ 0, ... 1, 0]]]), 'target_seq': Tensor(shape=[36, 5, 17], dtype=Int64, value= [[[2500, 4019, ... -1, -1]]]), 'gv_feat': Tensor(shape=[36, 1], dtype=Float64, value= [[ 0.00000000e+00],...[ 0.00000000e+00]]), 'att_feats': Tensor(shape=[36, 144, 1536], dtype=Float32, value= [[[ 1.78379081e-02, ... -9.93363326e-04]]]), 'att_mask': Tensor(shape=[36, 144], dtype=Float64, value= [[ 1.00000000e+00, ... 1.00000000e+00]])}但是运行报错:
[CRITICAL] ME(606029:281473551325760,MainProcess):2023-05-05-16:05:30.765.960 [mindspore/dataset/engine/datasets.py:2778] Uncaught exception: Traceback (most recent call last): File "/home/ma-user/work/PureT/main.py", line 416, in <module> trainer.train() File "/home/ma-user/work/PureT/main.py", line 350, in train loss, loss_info = self.model(self.rl_stage, indices, input_seq, target_seq, gv_feat, att_feats, att_mask) File "/home/ma-user/anaconda3/envs/MindSpore/lib/python3.7/site-packages/mindspore/nn/cell.py", line 586, in __call__ out = self.compile_and_run(*args) File "/home/ma-user/anaconda3/envs/MindSpore/lib/python3.7/site-packages/mindspore/nn/cell.py", line 964, in compile_and_run self.compile(*inputs) File "/home/ma-user/anaconda3/envs/MindSpore/lib/python3.7/site-packages/mindspore/nn/cell.py", line 937, in compile _cell_graph_executor.compile(self, *inputs, phase=self.phase, auto_parallel_mode=self._auto_parallel_mode) File "/home/ma-user/anaconda3/envs/MindSpore/lib/python3.7/site-packages/mindspore/common/api.py", line 1006, in compile result = self._graph_executor.compile(obj, args_list, phase, self._use_vm_mode()) TypeError: 'int' object is not iterable代码中的this.model如下:
class PureTTrainOneStepCell(nn.TrainOneStepCell): def __init__(self, network, optimizer, sens=1.0): super(PureTTrainOneStepCell, self).__init__(network, optimizer, sens) self.cast = ops.Cast() self.hyper_map = ops.HyperMap() def set_sens(self, value): self.sens = value def construct(self, rl_stage, indices, input_seq, target_seq, gv_feat, att_feats, att_mask): print(rl_stage) print('PureTTrainOneStepCell\n') weights = self.weights loss, loss_info = self.network(rl_stage, indices, input_seq, target_seq, gv_feat, att_feats, att_mask) grads = self.grad(self.network, weights)(rl_stage, indices, input_seq, target_seq, gv_feat, att_feats, att_mask, self.cast(ops.tuple_to_array((self.sens,)), ms.float32)) grads = self.hyper_map(ops.partial(utils.clip_grad, cfg.SOLVER.GRAD_CLIP_TYPE, cfg.SOLVER.GRAD_CLIP), grads) grads = self.grad_reducer(grads) self.optimizer(grads) return loss, loss_info