PyTorch：使用相同随机种子的不同训练精度答案

【问题标题】：PyTorch: Different training accuracies using same random seedPyTorch：使用相同随机种子的不同训练精度
【发布时间】：2019-10-19 14:17:45
【问题描述】：

我试图在每个 epoch 之后在整个训练集上评估我的模型。这就是我所做的：

torch.manual_seed(1)
model = ConvNet(num_classes=num_classes)
cost_fn = nn.CrossEntropyLoss()
optimizer = torch.optim.SGD(model.parameters(), lr=learning_rate)  

def compute_accuracy(model, data_loader):
    correct_pred, num_examples = 0, 0
    for features, targets in data_loader:
        logits = model(features)
        predicted_labels = torch.argmax(logits, 1)
        num_examples += targets.size(0)
        correct_pred += (predicted_labels == targets).sum()
    return correct_pred.float()/num_examples * 100

for epoch in range(num_epochs):
    model = model.train()
    for features, targets in train_loader:
        logits = model(features)
        cost = cost_fn(logits, targets)
        optimizer.zero_grad()
        cost.backward()
        optimizer.step()

    model = model.eval()
    print('Epoch: %03d/%03d training accuracy: %.2f%%' % (
          epoch+1, num_epochs, 
          compute_accuracy(model, train_loader)))

输出令人信服：

Epoch: 001/005 training accuracy: 89.08%
Epoch: 002/005 training accuracy: 90.41%
Epoch: 003/005 training accuracy: 91.70%
Epoch: 004/005 training accuracy: 92.31%
Epoch: 005/005 training accuracy: 92.95%

然后我在训练循环的末尾添加了另一行，以便在每个 epoch 之后在整个测试集上评估模型：

for epoch in range(num_epochs):
    model = model.train()
    for features, targets in train_loader:
        logits = model(features)
        cost = cost_fn(logits, targets)
        optimizer.zero_grad()
        cost.backward()
        optimizer.step()

    model = model.eval()
    print('Epoch: %03d/%03d training accuracy: %.2f%%' % (
          epoch+1, num_epochs, 
          compute_accuracy(model, train_loader)))
    print('\t\t testing accuracy: %.2f%%' % (compute_accuracy(model, test_loader)))

但训练精度开始发生变化：

Epoch: 001/005 training accuracy: 89.08%
         testing accuracy: 87.66%
Epoch: 002/005 training accuracy: 90.42%
         testing accuracy: 89.04%
Epoch: 003/005 training accuracy: 91.84%
         testing accuracy: 90.01%
Epoch: 004/005 training accuracy: 91.86%
         testing accuracy: 89.83%
Epoch: 005/005 training accuracy: 92.45%
         testing accuracy: 90.32%

我做错了吗？我希望训练精度保持不变，因为手动种子在两种情况下都是 1。这是预期的输出吗？

【问题讨论】：

如果 model.eval() 正在做与随机相关的事情，它会影响后面的所有随机数。将随机想象为在手动种子时确定的一些无限的随机位流。所有生成随机数的调用都使用相同的流。我认为在每次调用“train()”之前播种会给你想要的一致结果
我以为我在实现中做错了什么。但看起来它与随机数的行为有关。感谢您指出这一点。

标签： python validation pytorch random-seed

【解决方案1】：

random seed 的设置并没有阻止模型学习以获得更高的准确度，因为random seed 是伪随机数。在这种情况下，您已经告诉模型使用随机数（“1”）对训练数据进行混洗。

【讨论】：