使用mindspore.value_and_grad方法进行网络训练时报错
收藏回复举报
使用mindspore.value_and_grad方法进行网络训练时报错
t('forum.solved') 已解决
新人帖
发表于2024-01-09 20:49:17
0 查看

我正在实现将一个pytorch框架下的项目(已验证可以正常运行)转成mindspore框架下,在进行网络训练时,报了以下错误,请问这个应该怎么解决呢?有没有大佬解决过类似的问题? true Hardware Environment(Ascend/GPU/CPU) / 硬件环境: GPU cuda11.6

Software Environment / 软件环境 (Mandatory / 必填):

  • MindSpore version (e.g., 1.7.0.Bxxx) : 2.2.10
  • Python version (e.g., Python 3.7.5) : 3.9.18
  • OS platform and distribution (e.g., Linux Ubuntu 16.04): Linux Ubuntu 20.04 x86_64 GNU/Linux
  • GCC/Compiler version (if compiled from source): 8.5.0

预期结果是输出:

00 <class 'mindspore.common._stub_tensor.StubTensor'> <class 'mindspore.common._stub_tensor.StubTensor'> (50, 3, 32, 32) (50,)
train_step1
forward_fn1
forward_fn2
forward_fn3
train_step2
train_step3
11

而实际上只输出了前两行就没了

00 <class 'mindspore.common._stub_tensor.StubTensor'> <class 'mindspore.common._stub_tensor.StubTensor'> (50, 3, 32, 32) (50,)
train_step1

mindspore框架下的代码如下:

def _train(self, loader_tr, clf_group: dict, clf_name='train', soft_target=False, log_show=True):
        """Represents one epoch.

        :param loader_tr: (:obj:`torch.utils.data.DataLoader`) The training data wrapped in DataLoader.

        Accuracy and loss in the each iter will be recorded.

        """
        iter_out = self.args.out_iter_freq
        loss_list = []
        right_count_list = []
        samples_per_batch = []
        # clf_group['clf'].train()
        clf_group['clf'].set_train(True)

        optimizer = clf_group['optimizer']
        def train_step(data, label):
            print('train_step1')
            (loss, out), grads = grad_fn(data, label)
            print('train_step2')
            optimizer(grads)
            print('train_step3')
            return loss,out
        
        def forward_fn(data, label):
            print('forward_fn1')
            out, _, _ = self.clf(data)
            print('forward_fn2')
            loss = mindspore.ops.cross_entropy(out, label)
            print('forward_fn3')
            return loss, out
        
        grad_fn = mindspore.value_and_grad(forward_fn, None, optimizer.parameters, has_aux=True)

        for batch_idx, (x, y, _, _) in enumerate(loader_tr):
        
            squeeze = mindspore.ops.Squeeze(1)
            x = squeeze(x)
            if soft_target: 
                y = y.astype(np.float32)
            else:
                y = y.astype(np.int32)

            
            print('00',type(x),type(y),x.shape,y.shape)

            loss,out = train_step(x, y)
            print(11)
            
            
            loss_list.append(loss.item())
            
            if soft_target:
                pred_compare = out.max(axis=1, return_indices=True)[1]
                y_compare = y.max(axis=1, return_indices=True)[1]
                right_count_list.append((pred_compare == y_compare).sum().item())
                samples_per_batch.append(len(y))
            else:
                pred = out.max(axis=1, return_indices=True)[1]
                right_count_list.append((pred == y).sum().item())
                samples_per_batch.append(len(y))

以下是pytorch版的源码

def _train(self, loader_tr, clf_group: dict, clf_name='train', soft_target=False, log_show=True):
        """Represents one epoch.

        :param loader_tr: (:obj:`torch.utils.data.DataLoader`) The training data wrapped in DataLoader.

        Accuracy and loss in the each iter will be recorded.

        """
        iter_out = self.args.out_iter_freq
        loss_list = []
        right_count_list = []
        samples_per_batch = []
        clf_group['clf'].train()
        for batch_idx, (x, y, _, _) in enumerate(loader_tr):
            x, y = x.cuda(), y.cuda()
            clf_group['optimizer'].zero_grad()
            out, _, _ = self.clf(x)
            if soft_target:
                pred_compare = out.max(1)[1]
                y_compare = y.max(1)[1]
                right_count_list.append((pred_compare == y_compare).sum().item())
                samples_per_batch.append(len(y))
            else:
                pred = out.max(1)[1]
                right_count_list.append((pred == y).sum().item())
                samples_per_batch.append(len(y))
            loss = F.cross_entropy(out, y)
            loss_list.append(loss.item())
            loss.backward()
            clf_group['optimizer'].step()

        clf_group['scheduler'].step()

我要发帖子