TensorFlow2 - 模型子类化 ValueError

Question

我正在尝试使用 TensorFlow 2 的模型子分类创建 LeNet-300-100 密集神经网络。我的代码如下：

batch_size = 32
num_epochs = 20


# Load MNIST dataset-
(X_train, y_train), (X_test, y_test) = tf.keras.datasets.mnist.load_data()

X_train = X_train.astype('float32') / 255.0
X_test = X_test.astype('float32') / 255.0

# Convert class vectors/target to binary class matrices or one-hot encoded values-
y_train = tf.keras.utils.to_categorical(y_train, num_classes)
y_test = tf.keras.utils.to_categorical(y_test, num_classes)

X_train.shape, y_train.shape
# ((60000, 28, 28), (60000, 10))

X_test.shape, y_test.shape
# ((10000, 28, 28), (10000, 10)) 




class LeNet300(Model):
    def __init__(self, **kwargs):
        super(LeNet300, self).__init__(**kwargs)
        
        self.flatten = Flatten()
        self.dense1 = Dense(units = 300, activation = 'relu')
        self.dense2 = Dense(units = 100, activation = 'relu')
        self.op = Dense(units = 10, activation = 'softmax')

    def call(self, inputs):
        x = self.flatten(inputs)
        x = self.dense1(x)
        x = self.dense2(x)
        return self.op(x)




# Instantiate an object using LeNet-300-100 dense model-
model = LeNet300()

# Compile the defined model-
model.compile(
        optimizer=tf.keras.optimizers.Adam(),
        loss=tf.keras.losses.SparseCategoricalCrossentropy(),
        metrics=['accuracy']
        )


# Define early stopping callback-
early_stopping_callback = tf.keras.callbacks.EarlyStopping(
        monitor = 'val_loss', min_delta = 0.001,
        patience = 3)

# Train defined and compiled model-
history = model.fit(
    x = X_train, y = y_train,
    batch_size = batch_size, shuffle = True,
    epochs = num_epochs,
    callbacks = [early_stopping_callback],
    validation_data = (X_test, y_test)
    )

在调用“model.fit()”时，出现以下错误：

ValueError: Shape mismatch: 标签的形状（收到 (320,)） 应该等于 logits 的形状，除了最后一个维度 （收到 (32, 10)）。

怎么了？

谢谢

Answer 1

loss SparseCategoricalCrossentropy 不采用 one-hot encoding 来计算 loss。在文档中，他们提到了

当有两个或多个标签类别时使用此交叉熵损失函数。我们希望标签以整数形式提供。如果您想使用 one-hot 表示提供标签，请使用 CategoricalCrossentropy 损失。 y_pred 的每个特征应该有 # 个类浮点值，y_true 的每个特征应该有一个浮点值。

因此，您会收到错误消息。如果您观察堆栈跟踪，则损失函数中会出现错误，

    /home/ubuntu/.local/lib/python3.6/site-packages/tensorflow/python/keras/losses.py:1569 sparse_categorical_crossentropy
        y_true, y_pred, from_logits=from_logits, axis=axis)
    /home/ubuntu/.local/lib/python3.6/site-packages/tensorflow/python/util/dispatch.py:201 wrapper
        return target(*args, **kwargs)
    /home/ubuntu/.local/lib/python3.6/site-packages/tensorflow/python/keras/backend.py:4941 sparse_categorical_crossentropy
        labels=target, logits=output)
    /home/ubuntu/.local/lib/python3.6/site-packages/tensorflow/python/util/dispatch.py:201 wrapper
        return target(*args, **kwargs)
    /home/ubuntu/.local/lib/python3.6/site-packages/tensorflow/python/ops/nn_ops.py:4241 sparse_softmax_cross_entropy_with_logits_v2
        labels=labels, logits=logits, name=name)
    /home/ubuntu/.local/lib/python3.6/site-packages/tensorflow/python/util/dispatch.py:201 wrapper
        return target(*args, **kwargs)
    /home/ubuntu/.local/lib/python3.6/site-packages/tensorflow/python/ops/nn_ops.py:4156 sparse_softmax_cross_entropy_with_logits
        logits.get_shape()))

    ValueError: Shape mismatch: The shape of labels (received (320,)) should equal the shape of logits except for the last dimension (received (32, 10)).

我建议使用CategoricalCrossentropy。

Answer 2

这是因为第一个 Dense 层的输入应该被展平。 MNIST 数据的每个数字都有 28x28 的网格/图像。这个 28x28 的数据应该被展平为 784 个输入数字。

所以就在第一个Dense(...) 层之前插入Flatten() keras 层，即执行Flatten()(inputs)。

参考 Flatten 层的this doc。