Pytorch autograd 引发错误：“预期 isFloatingType(grads[i].type().scalarType()) 为真，但当 loss.backward() 为假”答案

【问题标题】：Pytorch autograd raising the error: "Expected isFloatingType(grads[i].type().scalarType()) to be true, but got false" when loss.backward()Pytorch autograd 引发错误：“预期 isFloatingType(grads[i].type().scalarType()) 为真，但当 loss.backward() 为假”
【发布时间】：2023-03-18 00:00:02
【问题描述】：

我正在实施对抗性训练。以下代码不起作用：

    for i, data in tqdm(enumerate(train_loader), total=len(train_loader), smoothing=0.9):

        pc1, pc2 = data
        pc1 = pc1.to(device).transpose(2, 1).contiguous().float()
        pc2 = pc2.to(device).transpose(2, 1).contiguous().float()

        # train the discriminator
        net_gen.eval()
        net_disc.train()

        i_odds = torch.arange(start=1, end=pc1.shape[2], step=2)
        ...

        # train the flow extractor
        net_gen.train()
        net_disc.eval()

        flow_pred = net_gen(pc1, pc2)
        pc_pred = pc1 + flow_pred

        z_pred = net_disc(pc1[:, :, i_odds], pc_pred[:, :, i_odds])
        loss_flow = - z_pred.mean()

        opt_gen.zero_grad()

        loss_flow.backward()

我在最后一行收到以下错误：Expected isFloatingType(grads[i].type().scalarType()) to be true, but got false.

那会是什么？据我所知，模型和所有张量都是 cuda.Float 类型。

That is the traceback:
Traceback (most recent call last):
  File "/home/zuanazzi/anaconda3/envs/tomtom2/lib/python3.6/pdb.py", line 1667, in main
    pdb._runscript(mainpyfile)
  File "/home/zuanazzi/anaconda3/envs/tomtom2/lib/python3.6/pdb.py", line 1548, in _runscript
    self.run(statement)
  File "/home/zuanazzi/anaconda3/envs/tomtom2/lib/python3.6/bdb.py", line 434, in run
    exec(cmd, globals, locals)
  File "<string>", line 1, in <module>
  File "/home/zuanazzi/PointCloudFlow/flownet3d_pytorch/self_train_adversarial.py", line 452, in <module>
    main()
  File "/home/zuanazzi/PointCloudFlow/flownet3d_pytorch/self_train_adversarial.py", line 445, in main
    train(args, net_flow, net_disc, train_loader, test_loader, boardio, textio)
  File "/home/zuanazzi/PointCloudFlow/flownet3d_pytorch/self_train_adversarial.py", line 309, in train
    train_stats = train_one_epoch(args, net_flow, net_disc, train_loader, opt_flow, opt_disc, writer, epoch)
  File "/home/zuanazzi/PointCloudFlow/flownet3d_pytorch/self_train_adversarial.py", line 227, in train_one_epoch
    loss_flow.backward()
  File "/home/zuanazzi/anaconda3/envs/tomtom2/lib/python3.6/site-packages/torch/tensor.py", line 195, in backward
    torch.autograd.backward(self, gradient, retain_graph, create_graph)
  File "/home/zuanazzi/anaconda3/envs/tomtom2/lib/python3.6/site-packages/torch/autograd/__init__.py", line 99, in backward
    allow_unreachable=True)  # allow_unreachable flag
RuntimeError: Expected isFloatingType(grads[i].type().scalarType()) to be true, but got false.  (Could this error message be improved?  If so, please report an enhancement request to PyTorch.)

最初在这里提出的问题：https://discuss.pytorch.org/t/expected-isfloatingtype-grads-i-type-scalartype-to-be-true-but-got-false/67840

编辑：似乎基于 PointNet++ 层的架构不能用作鉴别器，因为它们不会返回梯度。目前，这更像是一种理论而不是一种主张。

【问题讨论】：

标签： python deep-learning pytorch autograd generative-adversarial-network

【解决方案1】：

正如错误提示，loss_flow 的数据类型不是浮点数（可能是整数类型）。

你需要验证你的鉴别器的输出

z_pred = net_disc(pc1[:, :, i_odds], pc_pred[:, :, i_odds])

确实是一个浮点张量。

【讨论】：

我也是这么想的，在调试模式下我验证了(Pdb) z_pred.type() 'torch.cuda.FloatTensor'
@VictorZuanazzi 你需要检查z_pred.grad的数据类型。有没有可能你的判别器是不可区分的？
（修正）判别器在生成器之前进行训练。我为此使用了 FlowNet3D 架构，据我所知，这是可区分的。但是，我无法检查 z_pred.grad，因为在 loss_flow.backward() 之后就可以使用 grads 了，这正是事情发生的地方