Keras .fit 提供比手动 Tensorflow 更好的性能

Question

我是 TensorFlow 和 Keras 的新手。首先，我按照https://www.tensorflow.org/tutorials/quickstart/advanced 教程进行操作。我现在正在调整它以在 CIFAR10 而不是 MNIST 数据集上进行训练。我重新创建了这个模型https://keras.io/examples/cifar10_cnn/，并尝试在我自己的代码库中运行它。

从逻辑上讲，如果模型、批量大小和优化器都相同，那么两者的性能应该相同，但事实并非如此。我认为可能是我在准备数据时犯了一个错误。所以我将 keras 代码中的 model.fit 函数复制到我的脚本中，它仍然表现得更好。使用 .fit 可以在 25 个 epoch 内获得大约 75% 的准确率，而使用手动方法大约需要 60 个 epoch。使用 .fit 我还可以实现稍微更好的最大准确度。

我想知道的是：.fit 是否在幕后做一些优化训练的事情？我需要在代码中添加什么才能获得相同的性能？我在做一些明显错误的事情吗？

感谢您的宝贵时间。

主要代码：

import tensorflow as tf
from tensorflow import keras
import msvcrt
from Plotter import Plotter


#########################Configuration Settings#############################

BatchSize = 32
ModelName = "CifarModel"

############################################################################


(x_train, y_train), (x_test, y_test) = tf.keras.datasets.cifar10.load_data()

print("x_train",x_train.shape)
print("y_train",y_train.shape)
print("x_test",x_test.shape)
print("y_test",y_test.shape)

x_train, x_test = x_train / 255.0, x_test / 255.0

# Convert class vectors to binary class matrices.
y_train = keras.utils.to_categorical(y_train, 10)
y_test = keras.utils.to_categorical(y_test, 10)



train_ds = tf.data.Dataset.from_tensor_slices(
    (x_train, y_train)).batch(BatchSize)

test_ds = tf.data.Dataset.from_tensor_slices((x_test, y_test)).batch(BatchSize)


loss_object = tf.keras.losses.CategoricalCrossentropy(from_logits=True)
optimizer = tf.keras.optimizers.RMSprop(learning_rate=0.0001,decay=1e-6)

# Create an instance of the model
model = ModelManager.loadModel(ModelName,10)


train_loss = tf.keras.metrics.Mean(name='train_loss')
train_accuracy = tf.keras.metrics.CategoricalAccuracy(name='train_accuracy')

test_loss = tf.keras.metrics.Mean(name='test_loss')
test_accuracy = tf.keras.metrics.CategoricalAccuracy(name='test_accuracy')



########### Using this function I achieve better results ##################

model.compile(loss='categorical_crossentropy',
              optimizer=optimizer,
              metrics=['accuracy'])
model.fit(x_train, y_train,
              batch_size=BatchSize,
              epochs=100,
              validation_data=(x_test, y_test),
              shuffle=True,
              verbose=2)

############################################################################

########### Using the below code I achieve worse results ##################

@tf.function
def train_step(images, labels):
  with tf.GradientTape() as tape:
    predictions = model(images, training=True)
    loss = loss_object(labels, predictions)
  gradients = tape.gradient(loss, model.trainable_variables)
  optimizer.apply_gradients(zip(gradients, model.trainable_variables))

  train_loss(loss)
  train_accuracy(labels, predictions)

@tf.function
def test_step(images, labels):
  predictions = model(images, training=False)
  t_loss = loss_object(labels, predictions)

  test_loss(t_loss)
  test_accuracy(labels, predictions)

epoch = 0
InterruptLoop = False
while InterruptLoop == False:
  #Shuffle training data
  train_ds.shuffle(1000)
  epoch = epoch + 1
  # Reset the metrics at the start of the next epoch
  train_loss.reset_states()
  train_accuracy.reset_states()
  test_loss.reset_states()
  test_accuracy.reset_states()

  for images, labels in train_ds:
    train_step(images, labels)

  for test_images, test_labels in test_ds:
    test_step(test_images, test_labels)

  test_accuracy = test_accuracy.result() * 100
  train_accuracy = train_accuracy.result() * 100

  #Print update to console
  template = 'Epoch {}, Loss: {}, Accuracy: {}, Test Loss: {}, Test Accuracy: {}'
  print(template.format(epoch,
                        train_loss.result(),
                        train_accuracy ,
                        test_loss.result(),
                        test_accuracy))

  # Check if keyboard pressed
  while msvcrt.kbhit():
    char = str(msvcrt.getch())
    if char == "b'q'":
      InterruptLoop = True
      print("Stopping loop")

型号：

from tensorflow.keras.layers import Dense, Flatten, Conv2D, Dropout, MaxPool2D
from tensorflow.keras import Model

class ModelData(Model):
  def __init__(self,NumberOfOutputs):
    super(ModelData, self).__init__()
    self.conv1 = Conv2D(32, 3, activation='relu', padding='same', input_shape=(32,32,3))
    self.conv2 = Conv2D(32, 3, activation='relu')
    self.maxpooling1 = MaxPool2D(pool_size=(2,2))
    self.dropout1 = Dropout(0.25)
    ############################
    self.conv3 = Conv2D(64,3,activation='relu',padding='same')
    self.conv4 = Conv2D(64,3,activation='relu')
    self.maxpooling2 = MaxPool2D(pool_size=(2,2))
    self.dropout2 = Dropout(0.25)
    ############################
    self.flatten = Flatten()
    self.d1 = Dense(512, activation='relu')
    self.dropout3 = Dropout(0.5)
    self.d2 = Dense(NumberOfOutputs,activation='softmax')

  def call(self, x):
    x = self.conv1(x)
    x = self.conv2(x)
    x = self.maxpooling1(x)
    x = self.dropout1(x)
    x = self.conv3(x)
    x = self.conv4(x)
    x = self.maxpooling2(x)
    x = self.dropout2(x)
    x = self.flatten(x)
    x = self.d1(x)
    x = self.dropout3(x)
    x = self.d2(x)
    return x

Answer 1

为了社区的利益，在此处（回答部分）提及解决方案（即使它出现在评论中）。

在同一个Dataset 上，使用Keras Model.fit 与使用Model 构建的Model 的准确度可能不同，主要是在数据被打乱的情况下，因为当我们打乱数据时，数据之间的拆分训练和测试（或验证）会有所不同，从而导致两种情况（Keras 和 Tensorflow）中的训练和测试数据不同。

如果我们想在Keras 和Tensorflow 中观察相同数据集和相似架构的类似结果，我们可以Turn off Shuffling the Data。

希望这会有所帮助。快乐学习！