召回率和精度后的分类准确率答案

【问题标题】：Classification accuracy after recall and precision召回率和精度后的分类准确率
【发布时间】：2016-06-19 02:19:42
【问题描述】：

我只是想知道这是否是计算分类准确度的合法方法：

获得精确召回阈值
对于每个阈值，将连续的 y_scores 二值化
从列联表（混淆矩阵）计算它们的准确性

返回阈值的平均准确度

recall, precision, thresholds = precision_recall_curve(np.array(np_y_true), np.array(np_y_scores))
accuracy = 0
for threshold in thresholds:
    contingency_table = confusion_matrix(np_y_true, binarize(np_y_scores, threshold=threshold)[0])
    accuracy += (float(contingency_table[0][0]) + float(contingency_table[1][1]))/float(np.sum(contingency_table))

print "Classification accuracy is: {}".format(accuracy/len(thresholds))

【问题讨论】：

标签： python numpy machine-learning classification

【解决方案1】：

您正朝着正确的方向前进。混淆矩阵无疑是计算分类器准确性的正确起点。在我看来，您的目标是接收器操作特性。

在统计学中，接收者操作特征 (ROC) 或 ROC 曲线是一种图形图，它说明了二元分类器系统在其区分阈值发生变化时的性能。 https://en.wikipedia.org/wiki/Receiver_operating_characteristic

AUC（曲线下面积）是分类器性能的衡量标准。更多信息和解释可以在这里找到：

https://stats.stackexchange.com/questions/132777/what-does-auc-stand-for-and-what-is-it

http://mlwiki.org/index.php/ROC_Analysis

这是我的实现，欢迎您改进/评论：

def auc(y_true, y_val, plot=False):  
#check input
if len(y_true) != len(y_val):
    raise ValueError('Label vector (y_true) and corresponding value vector (y_val) must have the same length.\n')
#empty arrays, true positive and false positive numbers
tp = []
fp = []
#count 1's and -1's in y_true
cond_positive = list(y_true).count(1)
cond_negative = list(y_true).count(-1)
#all possibly relevant bias parameters stored in a list
bias_set = sorted(list(set(y_val)), key=float, reverse=True)
bias_set.append(min(bias_set)*0.9)

#initialize y_pred array full of negative predictions (-1)
y_pred = np.ones(len(y_true))*(-1)

#the computation time is mainly influenced by this for loop
#for a contamination rate of 1% it already takes ~8s to terminate
for bias in bias_set:
    #"lower values tend to correspond to label −1"
    #indices of values which exceed the bias
    posIdx = np.where(y_val > bias)
    #set predicted values to 1
    y_pred[posIdx] = 1
    #the following function simply calculates results which enable a distinction 
    #between the cases of true positive and  false positive
    results = np.asarray(y_true) + 2*np.asarray(y_pred)
    #append the amount of tp's and fp's
    tp.append(float(list(results).count(3)))
    fp.append(float(list(results).count(1)))

#calculate false positive/negative rate
tpr = np.asarray(tp)/cond_positive
fpr = np.asarray(fp)/cond_negative
#optional scatterplot
if plot == True:
    plt.scatter(fpr,tpr)
    plt.show()
#calculate AUC
AUC = np.trapz(tpr,fpr)

return AUC

【讨论】：