model.fit() 从不调用指定的自定义指标

Question

按照documentation，我正在尝试实现一个自定义指标，但是该指标永远不会被调用。我添加了在调用指标时会产生错误的完整性检查。您可以在 notebook 中找到完整的示例。

def my_metric_fn(y_true, y_pred):
    1 / 0
    while True:
        pass
    squared_difference = tf.square(y_true - y_pred)
    return tf.reduce_mean(squared_difference, axis=-1)  

但是，将指标传递给model.compile()后，代码运行没有问题

model.compile(optimizer=opt, metrics=[my_metric_fn])
history = model.fit(
train_dataset,
validation_data=validation_dataset,
epochs=epochs,
callbacks=[early_stopping]
)

我实际得到的：

Epoch 1/100
59/59 [==============================] - 27s 451ms/step - loss: 16.3928 - my_metric_fn: 0.0000e+00 - val_loss: 16.5252 - val_my_metric_fn: 0.0000e+00
Epoch 2/100
59/59 [==============================] - 25s 420ms/step - loss: 16.3508 - my_metric_fn: 0.0000e+00 - val_loss: 16.5316 - val_my_metric_fn: 0.0000e+00
Epoch 3/100
59/59 [==============================] - 25s 420ms/step - loss: 16.3420 - my_metric_fn: 0.0000e+00 - val_loss: 16.5372 - val_my_metric_fn: 0.0000e+00
Epoch 4/100
59/59 [==============================] - 25s 417ms/step - loss: 16.3365 - my_metric_fn: 0.0000e+00 - val_loss: 16.5287 - val_my_metric_fn: 0.0000e+00
Epoch 5/100
59/59 [==============================] - 25s 418ms/step - loss: 16.3251 - my_metric_fn: 0.0000e+00 - val_loss: 16.5271 - val_my_metric_fn: 0.0000e+00

Answer 1

我认为您在同时使用指标和add_loss 时遇到了这个bug。也许尝试将您的指标显式添加到您的自定义层：

def my_metric_fn(y_true, y_pred):
    squared_difference = tf.square(y_true - y_pred)
    return tf.reduce_mean(y_true, axis=-1)  

class CTCLayer(layers.Layer):
    def __init__(self, name=None):
        super().__init__(name=name)
        self.loss_fn = keras.backend.ctc_batch_cost

    def call(self, y_true, y_pred):
        # Compute the training-time loss value and add it
        # to the layer using `self.add_loss()`.
        batch_len = tf.cast(tf.shape(y_true)[0], dtype="int64")
        input_length = tf.cast(tf.shape(y_pred)[1], dtype="int64")
        label_length = tf.cast(tf.shape(y_true)[1], dtype="int64")

        input_length = input_length * tf.ones(shape=(batch_len, 1), dtype="int64")
        label_length = label_length * tf.ones(shape=(batch_len, 1), dtype="int64")

        loss = self.loss_fn(y_true, y_pred, input_length, label_length)
        self.add_loss(loss)
        self.add_metric(my_metric_fn(y_true, y_pred), 'my_metric_fn')

        # At test time, just return the computed predictions
        return y_pred

Epoch 1/100
15/59 [======>.......................] - ETA: 17s - loss: 34.6694 - my_metric_fn: 10.3142

或将其添加到模型的末尾：

opt = keras.optimizers.Adam()
# Compile the model and return
model.compile(optimizer=opt)
model.add_metric(my_metric_fn(...), 'my_metric_fn')