如何拆分训练和测试数据以确保训练和测试中都存在相同的因素组合？

Question

有没有办法将数据拆分为训练和测试，以便测试数据中的所有分类预测变量组合都出现在训练数据中？如果给定测试和训练大小指定的比例无法拆分数据，则这些级别不应包含在测试数据中。

假设我有这样的数据：

SAMPLE_DF <- data.frame("FACTOR1" = c(rep(letters[1:2], 8), "g", "g", "h", "i"),
                        "FACTOR2" = c(rep(letters[3:5], 2,), rep("z", 3), "f"),
                        "response" = rnorm(10,10,1),
                        "node" = c(rep(c(1,2),5)))
> SAMPLE_DF
   FACTOR1 FACTOR2  response node
1        a       c 10.334690    1
2        b       d 11.467605    2
3        a       e  8.935463    1
4        b       c 10.253852    2
5        a       d 11.067347    1
6        b       e 10.548887    2
7        a       z 10.066082    1
8        b       z 10.887074    2
9        a       z  8.802410    1
10       b       f  9.319187    2
11       a       c 10.334690    1
12       b       d 11.467605    2
13       a       e  8.935463    1
14       b       c 10.253852    2
15       a       d 11.067347    1
16       b       e 10.548887    2
17       g       z 10.066082    1
18       g       z 10.887074    2
19       h       z  8.802410    1
20       i       f  9.319187    2

在测试数据中，如果ac 的 FACTOR 1 和 2 组合在一起，那么这也将出现在训练数据中。所有其他可能的组合也是如此。

createDataPartition 只为一个级别执行此操作，但我希望它适用于所有级别。

Answer 1

您可以尝试使用 dplyr 删除仅出现一次的组合，因此只会出现在训练或测试集中，然后使用 CreateDataPartition 进行拆分：

数据

SAMPLE_DF <- data.frame("FACTOR1" = rep(letters[1:2], 10),
                         "FACTOR2" = c(rep(letters[3:5], 2,), rep("z", 4)),
                         "num_pred" = rnorm(10,10,1),
                         "response" = rnorm(10,10,1))

下面你使用dplyr来计算factor1和factor2的组合数。如果其中任何一个是 1，那么您将它们过滤掉：

library(dplyr)
 mydf <- 
 SAMPLE_DF %>%
   mutate(all = paste(FACTOR1,FACTOR2)) %>%
   group_by(all) %>%
   summarise(total=n()) %>%
   filter(total>=2)

以上只保留至少出现两次的因素1和2的组合

您根据上述保留的组合从SAMPLE_DF 中删除行：

SAMPLE_DF2 <- SAMPLE_DF[paste(SAMPLE_DF$FACTOR1,SAMPLE_DF$FACTOR2) %in% mydf$all,] 

最后你让createDataPartition 为你做拆分：

library(caret)
IND_TRAIN <- createDataPartition(paste(SAMPLE_DF2$FACTOR1,SAMPLE_DF2$FACTOR2))$Resample

 #train set
 A <- SAMPLE_DF2[ IND_TRAIN,]
 #test set
 B <- SAMPLE_DF2[-IND_TRAIN,]
 >identical(sort(paste(A$FACTOR1,A$FACTOR2)) , sort(paste(B$FACTOR1,B$FACTOR2)))
 [1] TRUE

正如您在同一行看到的，组合完全相同！

Answer 2

这是我汇总的一个函数，它执行此操作并将数据框拆分为包含具有相同级别的因子变量（列）的训练集和测试集（根据用户的百分比喜好）。

getTrainAndTestSamples_BalancedFactors <- function(data, percentTrain = 0.75, inSequence = F, seed = 0){
    set.seed(seed) # Set Seed so that same sample can be reproduced in future also
    sample <- NULL
    train <- NULL
    test<- NULL

    listOfFactorsAndTheirLevels <- lapply(Filter(is.factor, data), summary)
    factorContainingOneElement <- sapply(listOfFactorsAndTheirLevels, function(x){any(x==1)})

    if (any(factorContainingOneElement))
        warning("This dataframe cannot be reliably split into sets that contain Factors equally represented. At least one factor contains only 1 possible level.")
    else {
        # Repeat loop until all Factor variables have same levels on both train and test set
        repeat{
          # Now Selecting 'percentTrain' of data as sample from total 'n' rows of the data  
          if (inSequence)
            sample <- 1:floor(percentTrain * nrow(data))
          else
            sample <- sample.int(n = nrow(data), size = floor(percentTrain * nrow(data)), replace = F)    
  
          train <- data[sample, ]   # create train set
          test  <- data[-sample, ]  # create test set
  
          train_factor_only <- Filter(is.factor, train) # df containing only 'train' Factors as columns
          test_factor_only <- Filter(is.factor, test)   # df containing only 'test' Factors as columns
  
          haveFactorsWithExistingLevels <- NULL
          for (i in ncol(train_factor_only)){ # for each column (i.e. factor variable)
            names_train <- names(summary(train_factor_only[,i]))  # get names of all existing levels in this factor column for 'train'
            names_test <- names(summary(test_factor_only[,i]))    # get names of all existing levels in this factor column for 'test'
    
            symmetric_diff <- union(setdiff(names_train,names_test), setdiff(names_test,names_train)) # get the symmetric difference between the two factor columns (from 'train' and 'test')
            if (length(symmetric_diff) == 0)  # if no elements in the symmetric difference set then it means that both have the same levels present at least once
              haveFactorsWithExistingLevels <- c(haveFactorsWithExistingLevels, TRUE) # append logic TRUE
            else # if some elements in the symmetric difference set then it means that one of the two sets (train, test) has levels the other doesn't and it will eventually flag up when using function predict()
              haveFactorsWithExistingLevels <- c(haveFactorsWithExistingLevels, FALSE) # append logic FALSE
          }
  
          if(all(haveFactorsWithExistingLevels))
            break # break out of the repeat loop because we found a split that has factor levels existing in both 'train' and 'test' sets, for all Factor variables
        }
    }

    return (list( "train" = train, "test" = test))
}

像这样使用：

df <- getTrainAndTestSamples_BalancedFactors(some_dataframe)
df$train # this is your train set
df$test # this is your test set

...不再有来自 R 的烦人错误！

这绝对可以改进，我期待在下面的 cmets 中找到更有效的方法，但是，可以直接使用代码。

享受吧！