关于验证精度和召回率的 tensorflow ModelCheckpoint

Question

每当验证精度和召回率提高时，我都想检查点模型——这是在验证准确性和验证损失之上。所以我添加了以下内容：

checkPointPath = os.path.join(checkPointDir, 'cp-{epoch:03d}-{val_binary_accuracy:.3f}-{val_loss:.4f}-{val_precision:.3f}-{val_recall:.3f}.h5')  


valAccuracyCheckPointCallBack = tf.keras.callbacks.ModelCheckpoint(checkPointPath,
                                                                   monitor='val_binary_accuracy',
                                                                   save_freq='epoch',
                                                                   save_weights_only=False,
                                                                   save_best_only=True,
                                                                   verbose=1)
                
valLossCheckPointCallBack = tf.keras.callbacks.ModelCheckpoint(checkPointPath,
                                                               monitor='val_loss',
                                                               save_freq='epoch',
                                                               save_weights_only=False,
                                                               save_best_only=True,
                                                               verbose=1)
            
valPrecisionCheckPointCallBack = tf.keras.callbacks.ModelCheckpoint(checkPointPath,
                                                                    monitor='val_precision',
                                                                    save_freq='epoch',
                                                                    save_weights_only=False,
                                                                    save_best_only=True,
                                                                    verbose=1)
            
valRecallCheckPointCallBack = tf.keras.callbacks.ModelCheckpoint(checkPointPath,
                                                                 monitor='val_recall',
                                                                 save_freq='epoch',
                                                                 save_weights_only=False,
                                                                 save_best_only=True,
                                                                 verbose=1)
callBacks = [accuracyTrainingStopCB, valAccuracyCheckPointCallBack, valLossCheckPointCallBack, valPrecisionCheckPointCallBack, valRecallCheckPointCallBack]

在代码的其他地方，我将指标定义如下：

  model.compile(loss=tf.keras.losses.BinaryCrossentropy(),
                optimizer=tf.keras.optimizers.Adam(learning_rate=0.001),
                metrics=[tf.keras.metrics.BinaryAccuracy(name='binary_accuracy', threshold=0.9),
                         tf.keras.metrics.Precision(name='precision', thresholds=0.9),
                         tf.keras.metrics.Recall(name='recall', thresholds=0.9)
                        ],
                )

最后我在 fit() 方法中传递了 callBacks

history = model.fit(
        train_generator,
        epochs=1000,
        verbose=1,
        validation_data=validation_generator,
        validation_steps=8,
        callbacks=callBacks
    )

但是，在训练期间，我发现它不尊重 val_precision 和 val_recall 并且它们总是被报告为 0

Epoch 56/1000
126/128 [============================>.] - ETA: 0s - loss: 0.1819 - binary_accuracy: 0.9102 - precision: 0.9662 - recall: 0.8502
Epoch 56: val_binary_accuracy did not improve from 0.87500
            
Epoch 56: val_loss did not improve from 0.22489
            
Epoch 56: val_precision did not improve from 0.00000
            
Epoch 56: val_recall did not improve from 0.00000
128/128 [==============================] - 2s 18ms/step - loss: 0.1796 - binary_accuracy: 0.9116 - precision: 0.9668 - recall: 0.8525 - val_loss: 0.4248 - val_binary_accuracy: 0.7656 - val_precision: 0.8400 - val_recall: 0.6562
Epoch 57/1000
127/128 [============================>.] - ETA: 0s - loss: 0.2490 - binary_accuracy: 0.8868 - precision: 0.9456 - recall: 0.8209
Epoch 57: val_binary_accuracy did not improve from 0.87500
    
Epoch 57: val_loss did not improve from 0.22489
    
Epoch 57: val_precision did not improve from 0.00000
    
Epoch 57: val_recall did not improve from 0.00000
128/128 [==============================] - 2s 18ms/step - loss: 0.2473 - binary_accuracy: 0.8877 - precision: 0.9461 - recall: 0.8223 - val_loss: 0.2993 - val_binary_accuracy: 0.8516 - val_precision: 0.9245 - val_recall: 0.7656

我错过了什么？

EDIT-1：我注意到“检查点回调”中的精度和召回率都从 inf 开始，很快就会达到 0 并且不会进一步提高。那些不应该从 -inf 开始并像二进制精度一样继续上升吗？

在实例化回调时是否需要修复以定义初始值？

Answer 1

当我添加“mode='max'”参数时，问题得到解决。

valPrecisionCheckPointCallBack = tf.keras.callbacks.ModelCheckpoint(checkPointPath,
                                                                    mode='max', 
                                                                    monitor='val_precision',
                                                                    save_freq='epoch',
                                                                    save_weights_only=False,
                                                                    save_best_only=True,
                                                                    verbose=1)
            
valRecallCheckPointCallBack = tf.keras.callbacks.ModelCheckpoint(checkPointPath,
                                                                 mode='max',
                                                                 monitor='val_recall',
                                                                 save_freq='epoch',
                                                                 save_weights_only=False,
                                                                 save_best_only=True,
                                                                 verbose=1)

Answer 2

如何监控单个类的精度？ 比如我要监控class 0的精度