分类变量的值限制为 53

Question

我正在使用 R 编程语言。我正在尝试将“随机森林”（统计模型）拟合到我的数据中，但问题是：我的一个分类变量有超过 53 个类别 - 显然 R 中的“随机森林”包不允许用户有超过 53 个类别，这使我无法在我的模型中使用这个变量。理想情况下，我想使用这个变量。

为了说明这个例子，我创建了一个数据集（称为“数据”），其中一个变量有超过 53 个类别：

#load libraries
library(caret)
library(randomforest)
library(ranger)

#first data set

cat_var <- c("a","b", "c", "d", "e", "f", "g", "h", "i", "j", "k", "l", "m", "n", "o", "p", "q", "r", 
"s", "t", "u", "v", "w", "x", "y", "z", "aa", "bb", "cc", "dd", "ee", "ff",
 "gg", "hh", "ii", "jj", "kk", "ll", "mm", "nn", "oo", "pp", "qq", "rr", "ss", "tt", "uu", "vv", "ww", "xx", "yy", "zz", "aaa", "bbb")

var_1 <- rnorm(54,10,10)
var_2 <- rnorm(54, 5, 5)
var_3 <- rnorm(54, 6,18)

response <- c("a","b")
response <- sample(response, 54, replace=TRUE, prob=c(0.3, 0.7))

data_1 = data.frame(cat_var, var_1, var_2, var_3, response)
data_1$response = as.factor(data_1$response)
data_1$cat_var = as.factor(data_1$cat_var)

#second data set


cat_var <- c("a","b", "c", "d", "e", "f", "g", "h", "i", "j", "k", "l", "m", "n", "o", "p", "q", "r", 
"s", "t", "u", "v", "w", "x", "y", "z", "aa", "bb", "cc", "dd", "ee", "ff",
 "gg", "hh", "ii", "jj", "kk", "ll", "mm", "nn", "oo", "pp", "qq", "rr", "ss", "tt", "uu", "vv", "ww", "xx", "yy", "zz", "aaa", "bbb")

var_1 <- rnorm(54,10,10)
var_2 <- rnorm(54, 5, 5)
var_3 <- rnorm(54, 6,18)

response <- c("a","b")
response <- sample(response, 54, replace=TRUE, prob=c(0.3, 0.7))

data_2 = data.frame(cat_var, var_1, var_2, var_3, response)
data_2$response = as.factor(data_2$response)
data_2$cat_var = as.factor(data_2$cat_var)

# third data set


cat_var <- c("a","b", "c", "d", "e", "f", "g", "h", "i", "j", "k", "l", "m", "n", "o", "p", "q", "r", 
"s", "t", "u", "v", "w", "x", "y", "z", "aa", "bb", "cc", "dd", "ee", "ff",
 "gg", "hh", "ii", "jj", "kk", "ll", "mm", "nn", "oo", "pp", "qq", "rr", "ss", "tt", "uu", "vv", "ww", "xx", "yy", "zz", "aaa", "bbb")

var_1 <- rnorm(54,10,10)
var_2 <- rnorm(54, 5, 5)
var_3 <- rnorm(54, 6,18)

response <- c("a","b")
response <- sample(response, 54, replace=TRUE, prob=c(0.3, 0.7))

data_3 = data.frame(cat_var, var_1, var_2, var_3, response)
data_3$response = as.factor(data_3$response)
data_3$cat_var = as.factor(data_3$cat_var)

#combine data sets

data = rbind(data_1, data_2, data_3)

从这里开始，我对拟合随机森林模型感兴趣。我查看了不同的 stackoverflow 帖子（例如 R randomForest too many categories error even with fewer than 53 categories、R - Random Forest and more than 53 categories），这就是我注意到的。

如果您尝试按原样拟合随机森林模型，会发生以下情况：

#random forest using the "Randomforest" library

rf = randomForest(response ~ var_1 + var_2 + var_3 + cat_var, data=data, ntree=50, mtry=2)

Error in randomForest.default(m, y, ...) : 
  Can not handle categorical predictors with more than 53 categories.

在其中一篇文章中，一位用户建议使用“caret”库来拟合模型 - 显然 caret 模型没有 53 个类别的限制。这可行，但我不确定这是否正确：

#random forest using the "caret" and "ranger" libraries: (are these correct?)

random_forest <- train(response ~., 
                 data = data, 
                 method = 'ranger')

random_forest <- train(response ~., 
                 data = data, 
                 method = 'rf')

最后，另一位用户建议使用“模型矩阵”方法，但我不确定我是否完全理解了这种方法：

#model matrix method


dummyMat <- model.matrix(response ~ var_1 + var_2 + var_3 + cat_var, data, # set contrasts.arg to keep all levels
                         contrasts.arg = list(var_1 = contrasts(data$var_1, contrasts = T),  var_3 = contrasts(data$var_3, contrasts = T),  cat_var = contrasts(data$cat_var, contrasts = F)
                                             var_2 = contrasts(data$var2, contrasts = T))) 
data2 <- cbind(data, dummyMat[,c(4:ncol(dummyMat)]) # just removing intercept column

rf = randomForest(response ~ var_1 + var_2 + var_3 + cat_var, data=data2, ntree=50, mtry=2)

有人可以建议我如何解决这个问题吗？第二种方法（使用“插入符号”）是否正确？

谢谢

Answer 1

我可以告诉你caret 方法是正确的。 caret 包含用于数据拆分、预处理、特征选择和模型调整以及重采样交叉验证的工具。在这里，我发布了一个使用 caret 包拟合模型的典型工作流程（以您发布的数据为例）。

首先，我们设置了一种交叉验证方法来调整所选模型的超参数（在您的情况下，ranger 和 randomForest 的调整参数为 mtry，splitrule 和 min.node.size 为 @ 987654331@)。在示例中，我选择了 k=10 的k-fold corss-validation

library(caret)
control <- trainControl(method="cv",number = 10)

然后我们创建一个网格，其中包含要调整的参数可以采用的可能值

rangergrid <- expand.grid(mtry=2:(ncol(data)-1),splitrule="extratrees",min.node.size=seq(0.1,1,0.1))
rfgrid <- expand.grid(mtry=2:(ncol(data)-1))

最后，我们拟合所选模型：

random_forest_ranger <- train(response ~., 
                       data = data, 
                       method = 'ranger',
                       trControl=control,
                       tuneGrid=rangergrid)


random_forest_rf <- train(response ~., 
                       data = data, 
                       method = 'rf',
                       trControl=control,
                       tuneGrid=rfgrid)

train 函数的输出如下所示：

> random_forest_rf
Random Forest 

162 samples
  4 predictor
  2 classes: 'a', 'b' 

No pre-processing
Resampling: Cross-Validated (10 fold) 
Summary of sample sizes: 146, 146, 146, 145, 146, 146, ... 
Resampling results across tuning parameters:

  mtry  Accuracy   Kappa      
  2     0.6852941   0.00000000
  3     0.6852941   0.00000000
  4     0.6602941  -0.04499494

Accuracy was used to select the optimal model using the largest value.
The final value used for the model was mtry = 2.

有关caret 包的更多信息，请查看在线vignette