将 keras 模型架构转换为 Pytorch

Question

我正在尝试将 keras 模型转换为 pytorch 以进行人类活动识别。 keras 模型可以达到 98% 的准确率，而 Pytorch 模型只能达到 60% 的准确率。一直想不通，首先想到的是keras的padding=‘same’，但是我已经调整了pytorch的padding了。你能检查出什么问题吗？

keras代码如上

model = keras.Sequential()
model.add(layers.Input(shape=[100,12]))
model.add(layers.Conv1D(filters=32, kernel_size=3, padding="same"))
model.add(layers.BatchNormalization())
model.add(layers.ReLU())
model.add(layers.Conv1D(filters=64, kernel_size=3, padding="same"))
model.add(layers.BatchNormalization())
model.add(layers.ReLU())
model.add(layers.MaxPool1D(2))
model.add(layers.LSTM(64))
model.add(layers.Dense(units=128, activation='relu'))
model.add(layers.Dense(13, activation='softmax'))
model.summary()

我的pytorch模型代码如下

class CNN(nn.Module):
    def __init__(self):
        super(CNN, self).__init__()
        seq_len = 100
        # output: [, 32, 100]
        self.conv1 = nn.Conv1d(seq_len, 32, kernel_size=3, stride=1, padding=1)
        self.bn1 = nn.BatchNorm1d(32)
        # output: [, 64, 100]
        self.conv2 = nn.Conv1d(32, 64, kernel_size=3, padding=1)
        self.bn2 = nn.BatchNorm1d(64)
        # output: [, 64, 50]
        self.mp = nn.MaxPool1d(kernel_size=2, stride=2)
        # output: [, 64]
        self.lstm = nn.LSTM(6, 64, 1)
        # output: [, 128]
        self.fc1 = nn.Linear(64, 128)
        # output: [, 13]
        self.fc2 = nn.Linear(128, 13)
        
    def forward(self, x):
        x = self.conv1(x) 
        x = self.bn1(x)
        x = F.relu(x)

        x = self.conv2(x)
        x = self.bn2(x)
        x = F.relu(x)
        x = self.mp(x)
        
        out, _ = self.lstm(x)
        x = out[:, -1, :]
        
        x = self.fc1(x)
        x = F.relu(x)
        x = self.fc2(x)
        
        return x

Answer 1

由于写在 cmets 中太长了，我将其写在答案中，在测试您的 PyTorch 架构的形状与您提到的随机张量的形状之后：torch.randn(1, 100, 12)( NCH 格式）。

这是结果：

input= torch.Size([1, 100, 12])
1st Conv= torch.Size([1, 32, 12])
1st batchNorm= torch.Size([1, 32, 12])
1st relu= torch.Size([1, 32, 12])
2nd Conv= torch.Size([1, 64, 12])
2nd batchnorm= torch.Size([1, 64, 12])
2nd relu= torch.Size([1, 64, 12])
1st maxPool= torch.Size([1, 64, 6])
LSTM= torch.Size([1, 64])
1st FC= torch.Size([1, 128])
3rd relu= torch.Size([1, 128])
2nd FC= torch.Size([1, 13])

这是您的网络正在接收的 100 个通道，正如您在 Keras 中第一次卷积后在 cmets 中提到的那样，它是 [batch_size, 100, 32] 但在torch中它正在更改为 [batch_size,32,12]

改成这样：

def __init__(self):
        super(CNN, self).__init__()
        in_channels = 12
        # output: [, 32, 100]
        self.conv1 = nn.Conv1d(in_channels, 32, kernel_size=3, stride=1, padding=1)
        self.bn1 = nn.BatchNorm1d(32)
        # output: [, 64, 100]
        self.conv2 = nn.Conv1d(32, 64, kernel_size=3, padding=1)
        self.bn2 = nn.BatchNorm1d(64)
        # output: [, 64, 50]
        self.mp = nn.MaxPool1d(kernel_size=2, stride=2)
        # output: [, 64]
        self.lstm = nn.LSTM(50, 64, 1)
        # output: [, 128]
        self.fc1 = nn.Linear(64, 128)
        # output: [, 13]
        self.fc2 = nn.Linear(128, 13)
        self.softmax = nn.Softmax()

输出将是：

input= torch.Size([1, 12, 100])
1st Conv= torch.Size([1, 32, 100])
1st batchNorm= torch.Size([1, 32, 100])
1st relu= torch.Size([1, 32, 100])
2nd Conv= torch.Size([1, 64, 100])
2nd batchnorm= torch.Size([1, 64, 100])
2nd relu= torch.Size([1, 64, 100])
1st maxPool= torch.Size([1, 64, 50])
LSTM= torch.Size([1, 64])
1st FC= torch.Size([1, 128])
3rd relu= torch.Size([1, 128])
2nd FC= torch.Size([1, 13])