如何用两个或更多输入训练 Pytorch CNN答案

【问题标题】：How to train Pytorch CNN with two or more inputs如何用两个或更多输入训练 Pytorch CNN
【发布时间】：2020-08-17 01:03:26
【问题描述】：

我有一张大图像，图像中的多个事件会影响分类。我正在考虑将大图像分成小块，并从每个块中获取特征并将输出连接在一起进行预测。

我的代码是这样的：

train_load_1 = DataLoader(dataset=train_dataset_1, batch_size=100, shuffle=False)
train_load_2 = DataLoader(dataset=train_dataset_2, batch_size=100, shuffle=False)
train_load_3 = DataLoader(dataset=train_dataset_3, batch_size=100, shuffle=False)

test_load_1 = DataLoader(dataset=test_dataset_1, batch_size=100, shuffle=True)
test_load_2 = DataLoader(dataset=test_dataset_2, batch_size=100, shuffle=True)
test_load_3 = DataLoader(dataset=test_dataset_3, batch_size=100, shuffle=True)

class Net(nn.Module): 
   def __init__(self):
      super(Net, self).__init__()
      self.conv = nn.Conv2d( ... )  # set up your layer here
      self.fc1 = nn.Linear( ... )  # set up first FC layer
      self.fc2 = nn.Linear( ... )  # set up the other FC layer

   def forward(self, x1, x2, x3): 
      o1 = self.conv(x1)
      o2 = self.conv(x2)
      o3 = self.conv(x3)
      combined = torch.cat((o1.view(c.size(0), -1),
                            o2.view(c.size(0), -1),
                            o3.view(c.size(0), -1)), dim=1)
      out = self.fc1(combined)
      out = self.fc2(out)
      return F.softmax(x, dim=1)

model = Net().to(device)
optimizer = optim.SGD(model.parameters(), lr=0.01)

for epoch in epochs: 
   model.train()
   
   for batch_idx, (inputs, labels) in enumerate(train_loader_1): 
   **### I am stuck here, how to enumerate all three train_loader to pass input_1, input_2, input_3 into model and share the same label? Please note in train_loader I have set shuffle=False, this is to make sure train_loader_1, train_loader_2, train_loader_3 are getting the same label **

感谢您的帮助！

【问题讨论】：

标签： pytorch conv-neural-network

【解决方案1】：

您可以使用单个 dataLoader 元素代替使用 3 个单独的 dataLoader 元素，其中每个数据点包含图像的 3 个单独部分。

像这样：

dataLoader = [[[img1_part1],[img1_part2],[img1_part3], label1], [[img2_part1],[img2_part2],[img2_part3], label2]....]

这样你就可以在训练循环中使用它：

for img in dataLoader:
    part1,part2,part3,label = img
    out = model.forward(part1,part2,part3)
    loss = loss_fn(out, label)
    loss.backward()
    optimizer.step()

【讨论】：

感谢您的帮助。如何将图像加载到数据加载器中 [[[img1_part1],[img1_part2],[img1_part3], label1], [[img2_part1],[img2_part2],[img2_part3], label2]....]？
@Suddala Srujan 我对 Ling 有同样的问题：如何做到这一点？
好的，我看到下面的答案了，对不起

【解决方案2】：

对于具有该格式的图像部分：
您可以遍历图像并将它们附加到列表或 numpy 数组中。

def make_parts(full_image):
    # some code
    # returns a list of image parts after converting them into torch tensors
    return [TorchTensor_of_part1, TorchTensor_of_part2, TorchTensor_of_part3]

list_of_parts_and_labels = []
for image,label in zip(full_img_data, labels):
    parts = make_parts(image)
    list_of_parts_and_labels.append([parts, torch.tensor(label)])

如果您想将图像加载到 dataLoader 中，假设您已经拥有上述格式的图像部分和标签：

train_loader = torch.utils.data.DataLoader(list_of_parts_and_labels,
               shuffle = True, batch_size = BATCH_SIZE)

然后将其用作，

for data in train_loader:
    parts, label = data
    out = model.forward(*parts)
    loss = loss_fn(out, label)

【讨论】：