结合训练+测试数据并在R中运行交叉验证

Question

我有以下 R 代码，它在一组训练和测试数据上运行一个简单的 xgboost 模型，目的是预测二元结果。

我们从

开始

1) 阅读相关库。

library(xgboost)
library(readr)
library(caret)

2) 清理训练和测试数据

train.raw = read.csv("train_data", header = TRUE, sep = ",")
drop = c('column')
train.df = train.raw[, !(names(train.raw) %in% drop)]
train.df[,'outcome'] = as.factor(train.df[,'outcome'])


test.raw = read.csv("test_data", header = TRUE, sep = ",")
drop = c('column')
test.df = test.raw[, !(names(test.raw) %in% drop)]
test.df[,'outcome'] = as.factor(test.df[,'outcome'])

train.c1 = subset(train.df ,  outcome == 1)
train.c0 = subset(train.df ,  outcome == 0)

3) 对格式正确的数据运行 XGBoost。

train_xgb = xgb.DMatrix(data.matrix(train.df [,1:124]), label = train.raw[, "outcome"])
test_xgb = xgb.DMatrix(data.matrix(test.df[,1:124])) 

4) 运行模型

model_xgb = xgboost(data = train_xgb, nrounds = 8, max_depth = 5, eta = .1, eval_metric = "logloss", objective = "binary:logistic", verbose = 5)

5) 做出预测

pred_xgb <- predict(model_xgb, newdata = test_xgb)

我的问题是：我如何调整这个过程，以便我只是拉入/调整一个“训练”数据集，并获得对交叉验证文件的保留集的预测？

Answer 1

要在 xgboost 调用中指定 k-fold CV，需要使用 nfold = some integer 参数调用 xgb.cv，以保存每个重新采样的预测，使用 prediction = TRUE 参数。例如：

xgboostModelCV <- xgb.cv(data = dtrain, 
                         nrounds =  1688,
                         nfold = 5,
                         objective = "binary:logistic",
                         eval_metric= "auc",
                         metrics = "auc",
                         verbose = 1,
                         print_every_n = 50,
                         stratified = T,
                         scale_pos_weight = 2
                         max_depth = 6, 
                         eta = 0.01, 
                         gamma=0,
                         colsample_bytree =  1 ,
                         min_child_weight = 1,
                         subsample=  0.5 ,
                         prediction = T)

xgboostModelCV$pred #contains predictions in the same order as in dtrain.
xgboostModelCV$folds #contains k-fold samples

这是一个不错的选择超参数的函数

function(train, seed){
  require(xgboost)
  ntrees=2000
  searchGridSubCol <- expand.grid(subsample = c(0.5, 0.75, 1), 
                                  colsample_bytree = c(0.6, 0.8, 1),
                                  gamma=c(0, 1, 2),
                                  eta=c(0.01, 0.03),
                                  max_depth=c(4,6,8,10))
  aucErrorsHyperparameters <- apply(searchGridSubCol, 1, function(parameterList){

    #Extract Parameters to test
    currentSubsampleRate <- parameterList[["subsample"]]
    currentColsampleRate <- parameterList[["colsample_bytree"]]
    currentGamma <- parameterList[["gamma"]]
    currentEta =parameterList[["eta"]]
    currentMaxDepth =parameterList[["max_depth"]]
    set.seed(seed)

    xgboostModelCV <- xgb.cv(data = train, 
                             nrounds = ntrees,
                             nfold = 5,
                             objective = "binary:logistic",
                             eval_metric= "auc",
                             metrics = "auc",
                             verbose = 1,
                             print_every_n = 50,
                             early_stopping_rounds = 200,
                             stratified = T,
                             scale_pos_weight=sum(all_data_nobad[index_no_bad,1]==0)/sum(all_data_nobad[index_no_bad,1]==1),
                             max_depth = currentMaxDepth, 
                             eta = currentEta, 
                             gamma=currentGamma,
                             colsample_bytree = currentColsampleRate,
                             min_child_weight = 1,
                             subsample=  currentSubsampleRate) 


    xvalidationScores <- as.data.frame(xgboostModelCV$evaluation_log)
    #Save rmse of the last iteration
    auc=xvalidationScores[xvalidationScores$iter==xgboostModelCV$best_iteration,c(1,4,5)]
    auc=cbind(auc, currentSubsampleRate, currentColsampleRate, currentGamma, currentEta,  currentMaxDepth)
    names(auc)=c("iter", "test.auc.mean", "test.auc.std", "subsample", "colsample", "gamma", "eta", "max.depth")
    print(auc)
    return(auc)
  })
  return(aucErrorsHyperparameters)
}

您可以更改网格值和网格中的参数，以及损失/评估指标。它与caret 网格搜索提供的类似，但caret 不提供定义alpha、lambda、colsample_bylevel、num_parallel_tree... 网格搜索中的超参数的可能性，除了定义一个我觉得很麻烦的自定义功能。 Caret具有自动预处理、CV内自动上/下采样等优点。

在 xgb.cv 调用之外设置种子将为 CV 选择相同的折叠，但在每一轮中不会选择相同的树，因此您最终会得到不同的模型。即使您在 xgb.cv 函数调用中设置种子，也不能保证您最终会得到相同的模型，但机会更高（取决于线程、模型的类型..-我喜欢不确定性并发现对结果影响不大）。

Answer 2

您可以使用 xgb.cv 并设置 prediction = TRUE。