【问题标题】:weird ROC curve prediction奇怪的 ROC 曲线预测
【发布时间】:2016-07-03 23:56:51
【问题描述】:

我有来自 svm 模型的以下预测 (prediction_svm_linear),我想使用 R 中的 pROC 包绘制 ROC 曲线。我得到 AUC 100%,这是不可能的,因为基于混淆矩阵我没有有完美的预测。显然我遗漏了一些东西,可能我不完全理解 ROC 曲线是如何工作的,你能否向我解释一下为什么会发生这种情况?

Confusion Matrix and Statistics

      Reference
Prediction Cancer Normal
Cancer     11      0
Normal      3      5

           Accuracy : 0.8421          
             95% CI : (0.6042, 0.9662)
No Information Rate : 0.7368          
P-Value [Acc > NIR] : 0.2227          

              Kappa : 0.6587          
Mcnemar's Test P-Value : 0.2482          

        Sensitivity : 0.7857          
        Specificity : 1.0000          
     Pos Pred Value : 1.0000          
     Neg Pred Value : 0.6250          
         Prevalence : 0.7368          
     Detection Rate : 0.5789          
   Detection Prevalence : 0.5789          
   Balanced Accuracy : 0.8929          

   'Positive' Class : Cancer

这是我的代码:

library(pROC)
    testData_class = c(rep(c("Normal", "Cancer"), c(5, 14)))
    prediction_svm_linear = data.frame(Cancer = c(0.11766249, 0.04765463, 0.08749940, 0.01715765, 0.10755376, 0.28358435, 0.37478957, 0.90603193, 0.91077112, 0.68602820, 0.64783894, 0.67916187,0.38785763, 0.66440580, 0.51897036, 0.93484214, 0.91719866, 0.83239007, 0.63491027), Normal = c(0.88233751, 0.95234537, 0.91250060, 0.98284235, 0.89244624, 0.71641565, 0.62521043, 0.09396807, 0.08922888, 0.31397180, 0.35216106, 0.32083813,0.61214237, 0.33559420, 0.48102964, 0.06515786, 0.08280134, 0.16760993, 0.36508973))

    result.roc.model1 <-  roc(testData$class, prediction_svm_linear$Cancer, 
                            levels = rev(levels(testData$class)))


>result.roc.model1
Call:
roc.default(response = testData$class, predictor = prediction.prob.b5_svm_linear$Cancer,     levels = rev(levels(testData$class)))

Data: prediction.prob.b5_svm_linear$Cancer in 5 controls (testData$class Normal) < 14 cases (testData$class Cancer).
Area under the curve: 1

【问题讨论】:

  • 您的混淆矩阵基于什么阈值?你能展示你是如何生成它的吗?显然 ROC 曲线告诉你有一个更好的阈值......
  • 我没有为混淆矩阵设置任何阈值。这是代码:confusionMatrix(testData_class, prediction_svm_linear)
  • 来自插入符号包?
  • 另外请检查您的代码,它目前无法运行,因此很难确切知道发生了什么。

标签: roc


【解决方案1】:

根据您的评论,我怀疑您滥用了 caret 包中的 confusionMatrix 函数。根据文档,第二个因素应该是“a factor of classes to be used as the true results”,而您的评论表明您正在通过data.frame 进行连续预测。它不仅与所需的格式不同,而且还应该是您的第一个参数。

你应该改用这样的东西:

predictions <- ifelse(prediction_svm_linear$Cancer > 0.2, "Cancer", "Normal")
confusionMatrix(predictions, testData_class)

【讨论】:

    【解决方案2】:

    对不起,我可能让您感到困惑,但这里是所有信息

    二进制预测:

    prediction_svm = c("Normal", "Normal", "Normal", "Normal", "Normal", "Normal", "Normal", "Cancer", "Cancer", "Cancer", "Cancer", "Cancer", "Normal", "Cancer", "Cancer", "Cancer", "Cancer", "Cancer", "Cancer")

    基本事实:

    testData_class = c(rep(c("Normal", "Cancer"), c(5, 14)))

    概率预测

    prediction_svm_linear.prob = data.frame(Cancer = c(0.11766249, 0.04765463, 0.08749940, 0.01715765, 0.10755376, 0.28358435, 0.37478957, 0.90603193, 0.91077112, 0.68602820, 0.64783894, 0.67916187,0.38785763, 0.66440580, 0.51897036, 0.93484214, 0.91719866, 0.83239007, 0.63491027), Normal = c(0.88233751, 0.95234537, 0.91250060, 0.98284235, 0.89244624, 0.71641565, 0.62521043, 0.09396807, 0.08922888, 0.31397180, 0.35216106, 0.32083813,0.61214237, 0.33559420, 0.48102964, 0.06515786, 0.08280134, 0.16760993, 0.36508973))

    我正在用这个命令构建混淆矩阵:

    confusionMatrix(prediction_svm, testData$class)

    library(pROC)
        result.roc.model1 <-  roc(testData$class, prediction_svm_linear.prob$Cancer, 
                                levels = rev(levels(testData$class)))
    
    
    >result.roc.model1
    Call:
    roc.default(response = testData$class, predictor = prediction.prob.b5_svm_linear$Cancer,     levels = rev(levels(testData$class)))
    
    Data: prediction.prob.b5_svm_linear$Cancer in 5 controls (testData$class Normal) < 14 cases (testData$class Cancer).
    Area under the curve: 1
    
    
    >result.coords.model1 <- coords(  result.roc.model1, "best", best.method="closest.topleft",ret=c("threshold", "accuracy"))
    
    >result.coords.model1
    
    threshold  accuracy 
    0.2006234 1.0000000
    

    【讨论】:

      猜你喜欢
      • 2017-02-05
      • 1970-01-01
      • 2013-04-30
      • 2023-03-10
      • 1970-01-01
      • 2013-01-26
      • 2018-05-14
      • 2020-10-01
      • 2019-12-03
      相关资源
      最近更新 更多