R xgboost xgb.cv pred 值：最佳迭代还是最终迭代？

Question

我正在使用 xgb.cv 函数来网格搜索 xgboost 的 R 实现中的最佳超参数。将预测设置为 TRUE 时，它会为不折叠观察提供预测。假设您使用提前停止，预测是否对应于最佳迭代的预测，或者它们是最终迭代的预测？

Answer 1

CV 预测对应于最佳迭代 - 您可以使用“严格”的 early_stopping 值看到这一点，然后将预测与使用“最佳”迭代次数和“最终”迭代次数训练的模型进行比较，例如：

# Load minimum reproducible example
library(xgboost)
data(agaricus.train, package='xgboost')
data(agaricus.test, package='xgboost')
train <- agaricus.train
dtrain <- xgb.DMatrix(train$data, label=train$label)
test <- agaricus.test
dtest <- xgb.DMatrix(test$data, label=test$label)

# Perform cross validation with a 'strict' early_stopping
cv <- xgb.cv(data = train$data, label = train$label, nfold = 5, max_depth = 2,
             eta = 1, nthread = 4, nrounds = 10, objective = "binary:logistic",
             prediction = TRUE, early_stopping_rounds = 1)

# Check which round was the best iteration (the one that initiated the early stopping)
print(cv$best_iteration)
[1] 3

# Get the predictions
head(cv$pred)
[1] 0.84574515 0.15447612 0.15390711 0.84502697 0.09661318 0.15447612

# Train a model using 3 rounds (corresponds to best iteration)
trained_model <- xgb.train(data = dtrain, max_depth = 2,
              eta = 1, nthread = 4, nrounds = 3,
              watchlist = list(train = dtrain, eval = dtrain),
              objective = "binary:logistic")
# Get predictions
head(predict(trained_model, dtrain))
[1] 0.84625006 0.15353635 0.15353635 0.84625006 0.09530514 0.15353635

# Train a model using 10 rounds (corresponds to final iteration)
trained_model <- xgb.train(data = dtrain, max_depth = 2,
              eta = 1, nthread = 4, nrounds = 10,
              watchlist = list(train = dtrain, eval = dtrain),
              objective = "binary:logistic")
head(predict(trained_model, dtrain))
[1] 0.9884467125 0.0123147098 0.0050151693 0.9884467125 0.0008781737 0.0123147098

因此，来自 CV 的预测与迭代次数为“最佳”而非“最终”时的预测相同。