如何在 keras 中正确实现 cohen kappa 度量？答案

【问题标题】：How to correctly implement cohen kappa metric in keras?如何在 keras 中正确实现 cohen kappa 度量？
【发布时间】：2019-11-06 17:32:20
【问题描述】：

我尝试使用 tensorflow 为 keras (https://www.tensorflow.org/guide/keras/train_and_evaluate#custom_metrics) 提供的自定义指标指南为我的项目实施 cohen kappa 指标。

在我训练我的模型并预测输出后，我将结果与sklearn.metrics.cohen_kappa_score 进行了比较，结果不一样。谁能告诉如何解决这个问题？代码如下：

class BinaryKappa(keras.metrics.Metric):
    """Stateful Metric to calculate kappa over all batches.
    Assumes predictions and targets of shape `(samples, 1)`.
    # Arguments
        name: String, name for the metric.
    """

    def __init__(self, name='glo_kappa', **kwargs):
        super(BinaryKappa, self).__init__(name=name, **kwargs)
        self.true_positives = self.add_weight(name='tp',initializer='zeros')
        self.true_negative = self.add_weight(name='tn',initializer='zeros')
        self.false_positives = self.add_weight(name='fp',initializer='zeros')
        self.false_negative = self.add_weight(name='fn',initializer='zeros')

    def reset_states(self):
        self.true_positives.assign(0.)
        self.true_negative.assign(0.)
        self.false_positives.assign(0.)
        self.false_negative.assign(0.)

    def update_state(self, y_true, y_pred, sample_weight=None):
        y_true = K.cast(y_true, 'int32')
        y_pred = K.cast(K.round(y_pred), 'int32')
        true_pos = K.cast(K.sum(y_pred * y_true),'int32')
        true_neg = K.cast(K.sum((1 - y_pred) * (1 - y_true)),'int32')
        false_pos = K.cast(K.sum(y_pred * (1 - y_true)),'int32')
        false_neg = K.cast(K.sum((1 - y_pred) * y_true),'int32')

        true_pos = K.cast(true_pos, "float32")
        true_neg = K.cast(true_neg, "float32")
        false_pos = K.cast(false_pos, "float32")
        false_neg = K.cast(false_neg, "float32")

        self.true_positives.assign_add(true_pos)
        self.true_negative.assign_add(true_neg)
        self.false_positives.assign_add(false_pos)
        self.false_negative.assign_add(false_neg)

    def result(self):
        sm = self.true_positives + self.true_negative + self.false_positives + self.false_negative
        obs_agree = (self.true_positives + self.true_negative) / sm
        poss_pos = (self.true_positives + self.false_negative) * (self.true_positives + self.false_positives) / (sm**2)
        poss_neg = (self.true_negative + self.false_negative) * (self.true_negative + self.false_positives) / (sm**2)
        poss_agree = poss_pos + poss_neg
        return (obs_agree - poss_agree) / (1 - poss_agree + K.epsilon())

【问题讨论】：

标签： python tensorflow keras scikit-learn

【解决方案2】：

使用来自 confuse_matrix 的信息执行此操作，不要注意其他一些 sklearn 库。

从 sklearn.metrics 导入混淆矩阵

NBC = NBC.fit(X_train,y_train)
cm = confusion_matrix(y_test, NBC.predict(X_test))
tn, fp, fn, tp = cm.ravel()
print('tn: ',tn)
print('fp: ',fp)
print('fn: ',fn)
print('tp: ',tp)
print('------------------')
print(cm) 

p_0 = (tn+??)/(tn+fp+fn+??)
print('p_0:',p_0)

P_a = ((tn+fp)/(tn+fp+fn+??))*((tn+fn)/(tn+fp+fn+??))
print('P_yes: ',P_yes)

P_b = ((fn+??)/(tn+fp+fn+??))*((fp+??)/(tn+fp+fn+??))
print('P_no: ',P_no)


pe = P_a + P_b
print('pe: ',pe)

κ = (p_0-pe)/(1-pe)
print('κ: ',κ)

【讨论】：