TensorFlow/Keras 使用特定类召回作为稀疏分类交叉熵的度量

Question

*底部更新

我正在尝试使用 3 个类中的 2 个作为度量标准，因此 A、B、C 类中的 B 类和 C 类。

（其原始性质是我的模型在类中高度不平衡 [~90% 是 A 类]，因此当我使用准确度时，每次预测 A 类时我得到 ~90% 的结果）

model.compile(
              loss='sparse_categorical_crossentropy', #or categorical_crossentropy
              optimizer=opt,
              metrics=[tf.keras.metrics.Recall(class_id=1, name='recall_1'),tf.keras.metrics.Recall(class_id=2, name='recall_2')]
              )

history = model.fit(train_x, train_y, batch_size=BATCH, epochs=EPOCHS, validation_data=(validation_x, validation_y), callbacks=[tensorboard, checkpoint])

这吐出了一个错误：

raise ValueError("Shapes %s and %s are incompatible" % (self, other))

ValueError: Shapes (None, 3) and (None, 1) are incompatible

模型摘要为：

Model: "sequential"
_________________________________________________________________
Layer (type)                 Output Shape              Param #
=================================================================
lstm (LSTM)                  (None, 120, 32)           19328
_________________________________________________________________
dropout (Dropout)            (None, 120, 32)           0
_________________________________________________________________
batch_normalization (BatchNo (None, 120, 32)           128
_________________________________________________________________
lstm_1 (LSTM)                (None, 120, 32)           8320
_________________________________________________________________
dropout_1 (Dropout)          (None, 120, 32)           0
_________________________________________________________________
batch_normalization_1 (Batch (None, 120, 32)           128
_________________________________________________________________
lstm_2 (LSTM)                (None, 32)                8320
_________________________________________________________________
dropout_2 (Dropout)          (None, 32)                0
_________________________________________________________________
batch_normalization_2 (Batch (None, 32)                128
_________________________________________________________________
dense (Dense)                (None, 32)                1056
_________________________________________________________________
dropout_3 (Dropout)          (None, 32)                0
_________________________________________________________________
dense_1 (Dense)              (None, 3)                 99
=================================================================
Total params: 37,507
Trainable params: 37,315
Non-trainable params: 192

请注意，如果使用，该模型可以正常工作，没有错误：

metrics=['accuracy']

但是this 和this 让我觉得有些东西没有按照 tf.metrics.SparseCategorical 的方式实现Recall()

来自

tf.metrics.SparseCategoricalAccuracy()

---

所以我转向了一个自定义指标，该指标陷入了其他问题的困境，因为我在类和装饰器方面非常文盲。

我从一个自定义指标示例中将其拼凑在一起（我不知道如何使用 sample_weight，所以我将其注释掉以备后用）：

class RelevantRecall(tf.keras.metrics.Metric):

    def __init__(self, name="Relevant_Recall", **kwargs):
        super(RelevantRecall, self).__init__(name=name, **kwargs)
        self.joined_recall = self.add_weight(name="B/C Recall", initializer="zeros")

    def update_state(self, y_true, y_pred, sample_weight=None):
        y_pred = tf.argmax(y_pred, axis=1)
        report_dictionary = classification_report(y_true, y_pred, output_dict = True)

        # if sample_weight is not None:
        #     sample_weight = tf.cast(sample_weight, "float32")
        #     values = tf.multiply(values, sample_weight)
        # self.joined_recall.assign_add(tf.reduce_sum(values))

        self.joined_recall.assign_add((float(report_dictionary['1.0']['recall'])+float(report_dictionary['2.0']['recall']))/2)
 
    def result(self):
        return self.joined_recall

    def reset_states(self):
        # The state of the metric will be reset at the start of each epoch.
        self.joined_recall.assign(0.0)


model.compile(
              loss='sparse_categorical_crossentropy', #or categorical_crossentropy
              optimizer=opt,
              metrics=[RelevantRecall()]
              )


history = model.fit(train_x, train_y, batch_size=BATCH, epochs=EPOCHS, validation_data=(validation_x, validation_y), callbacks=[tensorboard, checkpoint])

这个目标是返回一个[recall(b)+recall(c)/2] 的指标。我想像metrics=[recall(b),recall(c)] 那样分别返回两次召回会更好，但无论如何我无法让前者工作。

我收到一个张量布尔错误：OperatorNotAllowedInGraphError: using a 'tf.Tensor' as a Python 'bool' is not allowed: AutoGraph did convert this function. This might indicate you are trying to use an unsupported feature.，我在谷歌搜索后添加了：@tf.function 在我的自定义指标类上方。

这导致了新旧类类型错误：

super(RelevantRecall, self).__init__(name=name, **kwargs)
TypeError: super() argument 1 must be type, not Function

由于类有一个对象，我没有看到我是如何实现的？

正如我所说，我对这方面的所有方面都很陌生，因此对于如何使用仅选择一个预测类的度量来实现（以及如何最好地实现）的任何帮助将不胜感激。

或

如果我完全错了，请告诉我/指导我找到正确的资源

理想情况下，我想采用以前使用tf.keras.metrics.Recall(class_id=1.... 的方法，因为如果它有效的话，这似乎是最简洁的方法。

当在模型的回调部分中使用类似的函数时，我能够获得每个类的召回，但这似乎更加密集，因为我必须在每个结束时对 val/test 数据进行 model.predict时代。 也不清楚这是否甚至告诉模型专注于改进所选类（即在度量与回调中实现它的差异）

---

回调代码：

class MetricsCallback(Callback):
    def __init__(self, test_data, y_true):
        # Should be the label encoding of your classes
        self.y_true = y_true
        self.test_data = test_data

    def on_epoch_end(self, epoch, logs=None):
        # Here we get the probabilities - longer process
        y_pred = self.model.predict(self.test_data)

        # Here we get the actual classes
        y_pred = tf.argmax(y_pred,axis=1)
        report_dictionary = classification_report(self.y_true, y_pred, output_dict = True)
        print ("\n")
  
        print (f"Accuracy: {report_dictionary['accuracy']} - Holds: {report_dictionary['0.0']['recall']} - Sells: {report_dictionary['1.0']['recall']} - Buys: {report_dictionary['2.0']['recall']}")
        self._data = (float(report_dictionary['1.0']['recall'])+float(report_dictionary['2.0']['recall']))/2
        return

metrics_callback = MetricsCallback(test_data = validation_x, y_true = validation_y)

history = model.fit(train_x, train_y, batch_size=BATCH, epochs=EPOCHS, validation_data=(validation_x, validation_y), callbacks=[tensorboard, checkpoint, metrics_callback) 

---

更新 19/07/2021

• 我已将categorical_crossentropy 用于loss 而不是sparse_categorical_crossentropy。
• 对我的类/目标数组进行一次热编码。
• 使用 tf 召回：[tf.keras.metrics.Recall(class_id=1, name='recall_1')

我现在正在使用下面的代码。

train_y = tf.one_hot(train_y, 3)
validation_y = tf.one_hot(validation_y, 3)
test_y = tf.one_hot(test_y, 3)

model.compile(
    loss='categorical_crossentropy',
    optimizer=opt,
    metrics=[tf.keras.metrics.Recall(class_id=1, name='No'),tf.keras.metrics.Recall(class_id=2, name='Yes')]
    ) #tf.keras.metrics.Recall(class_id=0, name='Wait')

history = model.fit(train_x, train_y, batch_size=BATCH, epochs=EPOCHS, validation_data=(validation_x, validation_y), callbacks=[tensorboard, checkpoint])

感谢Abhishek Prajapat

这实现了相同的总体目标，并且由于少量互斥类可能对性能的差异/影响非常小，

但是在大量互斥类的情况下，我仍然没有解决方案来使用sparse_categorical_crossentropy实现与上述相同的目标

Answer 1

你的问题很简单。我为你整理了一个例子：

import tensorflow as tf
from sklearn.datasets import make_classification

data = make_classification(n_samples=1000, n_features=20, n_classes=3, n_clusters_per_class=1)

model = tf.keras.Sequential([
    tf.keras.layers.InputLayer(input_shape=(20)),
    tf.keras.layers.Dense(3, activation='softmax')
])

model.compile(
              loss=tf.keras.losses.CategoricalCrossentropy(), #or categorical_crossentropy
              optimizer='adam',
              metrics = [tf.keras.metrics.Recall(class_id=1)]
              )

y = tf.keras.utils.to_categorical(data[1], num_classes=3)

dataset = tf.data.Dataset.from_tensor_slices((data[0], y))
dataset = dataset.batch(10)

model.fit(dataset, epochs=10)

现在您可以看到，当您使用带有特定类 ID 的 metrics.Recall 时，您的输入 y 应该是一次性编码的。因此，如果我们有 3 个类，那么对于 0，它应该是 -> [1, 0, 0] 等等 1 -> [0, 1, 0] 和 2 -> [0, 0, 1]。

不使用额外的内存

import tensorflow as tf
from sklearn.datasets import make_classification

data = make_classification(n_samples=1000, n_features=20, n_classes=3, n_clusters_per_class=1)

model = tf.keras.Sequential([
    tf.keras.layers.InputLayer(input_shape=(20)),
    tf.keras.layers.Dense(3, activation='softmax')
])

model.compile(
              loss=tf.keras.losses.CategoricalCrossentropy(), #or categorical_crossentropy
              optimizer='adam',
              metrics = [tf.keras.metrics.Recall(class_id=1)]
              )

def encode(x, y):
    y = tf.one_hot(y, 3) # Here 3 is the number of classes
    return x, y

dataset = tf.data.Dataset.from_tensor_slices((data[0], data[1]))
dataset = dataset.map(encode)
dataset = dataset.batch(10)

model.fit(dataset, epochs=10)

新示例 -

import numpy as np
import tensorflow as tf
from sklearn.datasets import make_classification

data = make_classification(n_samples=1000, n_features=20, n_classes=3, n_clusters_per_class=1)

model = tf.keras.Sequential([
    tf.keras.layers.InputLayer(input_shape=(20)),
    tf.keras.layers.Dense(3, activation='softmax')
])

def encode(x, y):
    y = tf.one_hot(y, 3)
    return x, y

dataset = tf.data.Dataset.from_tensor_slices((data[0], data[1]))
dataset = dataset.map(encode)
dataset = dataset.batch(10)

m1 = tf.keras.metrics.Recall()
m2 = tf.keras.metrics.Recall()

def my_recall(y_true, y_pred):
    
    actual_a = y_true[:, 1]
    pred_a = y_pred[:, 1]
    
    actual_b = y_true[:, 2]
    pred_b = y_pred[:, 2]
    
    m1.update_state(actual_a, pred_a)
    m2.update_state(actual_b, pred_b)
    
    return (m1.result() + m2.result())/2

model.compile(
              loss=tf.keras.losses.CategoricalCrossentropy(), #or categorical_crossentropy
              optimizer='adam',
              metrics = [my_recall]
              )

model.fit(dataset, epochs=10)

为您更新的问题-

import numpy as np
import tensorflow as tf
from sklearn.datasets import make_classification

data = make_classification(n_samples=1000, n_features=20, n_classes=3, n_clusters_per_class=1)

model = tf.keras.Sequential([
    tf.keras.layers.InputLayer(input_shape=(20)),
    tf.keras.layers.Dense(3, activation='softmax')
])

dataset = tf.data.Dataset.from_tensor_slices((data[0], data[1]))
dataset = dataset.batch(10)

m1 = tf.keras.metrics.Recall()
m2 = tf.keras.metrics.Recall()

def my_recall(y_true, y_pred):
    y_true = tf.cast(y_true, dtype=tf.int32)
    actual_onehot = tf.one_hot(y_true, 3)
    actual_a = actual_onehot[1]
    pred_a = tf.reshape(y_pred[1], (1,3))
    actual_b = actual_onehot[2]
    pred_b = tf.reshape(y_pred[2], (1,3))  
    m1.update_state(actual_a, pred_a)
    m2.update_state(actual_b, pred_b)   
    return (m1.result() + m2.result())/2

model.compile(
              loss=tf.keras.losses.SparseCategoricalCrossentropy(),
              optimizer='adam',          
              metrics = [my_recall]
              )

model.fit(dataset, epochs=10)