在保留所有列的同时删除重复的全向组合

Question

我需要删除组 (ID) 中两列（feedID 和 feedID2）的重复组合，同时在数据集中保留大量其他列。应删除所有重复的行，无论是第 2 列中的 A 和第 3 列中的 B，反之亦然。 此外，我想删除所有行，例如两列中都有 A，或者其中一列中有 NA。 我无法对列之间的数据进行排序，即如果 A 在第 nr 2 列中，它应该保留在第 nr 2 列中。

我知道这可能是一个重复的问题，但其他答案似乎都不适用于我的数据集，或者要求相同的问题。 例如。 Finding unique combinations irrespective of position Removing duplicate combinations in R (irrespective of order)

 test <- data.frame(ID= c("49V", "49V","49V", "49V", "49V", "52V", "52V", "52V"),
                      feedID = c("A1", "A1", "G2", "A1", "G2", "B1", "D1",  "D2" ),
                    feedID2 = c("A1", "G2", "A1", "G2", "NA", "D1", "D2",  "NA" ))

 desiredoutput <- data.frame(ID= c("49V", "52V", "52V"),
                      feedID = c("A1","B1", "D1" ),
                    feedID2 = c("G2", "D1", "D2" ))

如果在不同的列中，以下代码不会删除重复项

   test2 <- test [!duplicated(test[,c("ID","feedID", "feedID2")]),]

这段代码什么都不做，但不会抛出任何错误

  test2 <-  test%>% distinct(1,2,3) # where numbers refer to the columns

此代码会产生一个错误，对于 dimnames，不确定这意味着什么。我的测试数据没有得到这个，我不确定为什么并且无法重现错误...

  indx <- !duplicated(t(apply(test, 1, sort))) # finds non - duplicates in sorted rows
   test[indx, ] 

有什么想法吗？

Answer 1

您的数据再次出现，但 "NA" 更改为 NA 和 stringsAsFactors=F

test <- data.frame(ID= c("49V", "49V","49V", "49V", "49V", "52V", "52V", "52V"),
                   feedID = c("A1", "A1", "G2", "A1", "G2", "B1", "D1",  "D2" ),
                   feedID2 = c("A1", "G2", "A1", "G2", NA, "D1", "D2",  NA ),
                   stringsAsFactors=F)

 library(dplyr)
 test %>% 
  filter(complete.cases(.)) %>%             # Remove rows with NA
  rowwise() %>%                             # Perform next step by row
  mutate(dup=paste0(sort(c(feedID,feedID2)),collapse="")) %>%   # Sort and combine feedID and feedID2
  ungroup() %>%
  group_by(ID) %>%                             # Remove rowwise grouping
  mutate(dup=duplicated(dup)) %>%           # Find duplicated feedID:feedID2 pairs
  filter(dup==F) %>%                        # Remove duplicated pairs
  filter(!(feedID==feedID2)) %>%            # Remove where feedID == feedID2
  select(-dup)                              # Remove dummy column


     ID feedID feedID2
1   49V     A1      G2
2   52V     B1      D1
3   52V     D1      D2

如果你只想在feedID和feedID2中寻找NA

将filter(complete.cases(.)) 替换为filter(!is.na(feedID) & !is.na(feedID2))

Answer 2

这是一个基本解决方案，使用 complete.cases 函数，并创建一个排序的 feedID 列：

# remove any rows with NA values
test <- test[complete.cases(test[,c('ID', 'feedID','feedID2')]),]
#remove any rows with feedID == feedID2
test <- test[!(test$feedID == test$feedID2),]
# add new feedID3 column
test$feedID3 <- apply(test, 1, function(x) paste(sort(c(x[2], x[3])), collapse = '-'))
# remove any duplicates, and remove last column
test[!duplicated(test[,c('feedID3', 'ID')]), -4]


   ID feedID feedID2
2 49V     A1      G2
6 52V     B1      D1
7 52V     D1      D2

数据

请注意，我们已将"NA" 转换为NA，并且我们还设置了stringsAsFactors = TRUE

test <- data.frame(ID= c("49V", "49V","49V", "49V", "49V", "52V", "52V", "52V"),
                   feedID = c("A1", "A1", "G2", "A1", "G2", "B1", "D1",  "D2" ),
                   feedID2 = c("A1", "G2", "A1", "G2", NA, "D1", "D2",  NA ),
                   stringsAsFactors = FALSE)

Answer 3

将“NA”更改为 NA，并设置 stringsAsFactors = F

library(dplyr)
library(stringr)

test <- data.frame(ID= c("49V", "49V","49V", "49V", "49V", "52V", "52V", "52V"),
                   feedID = c("A1", "A1", "G2", "A1", "G2", "B1", "D1",  "D2" ),
                   feedID2 = c("A1", "G2", "A1", "G2", NA, "D1", "D2",  NA ),
                   stringsAsFactors = F)

desiredoutput <- data.frame(ID= c("49V", "52V", "52V"),
                            feedID = c("A1","B1", "D1" ),
                            feedID2 = c("G2", "D1", "D2" ),
                            stringsAsFactors = F)

test %>% 
  # Remove NAs and all rows where the IDs are equal
  filter(!is.na(feedID),                       
         !is.na(feedID2),                      
         feedID != feedID2) %>%                
  # Group rowwise and create a sorted pair of the two ID columns
  rowwise() %>%                                
  mutate(revCheck = str_c(str_sort(c(feedID, feedID2)), collapse = "")) %>% 
  ungroup() %>% 
  # Find distinct ID pairs and keep all variables
  distinct(revCheck,
           .keep_all = T) %>% 
  # Find distinct rows for each ID pair. I kept these separate because I
  # think that's what you're asking for in your example, you want all
  # duplicates in feedID and all duplicates in feedID2 removed, not just
  # duplicate combinations of feedID and feedID2. See .keep_all in ?distinct
  distinct(feedID,
           .keep_all = T) %>% 
  distinct(feedID2,
           .keep_all = T) %>% 
  # Remove the sorted pair id
  select(-revCheck) %>% 
  # Return a dataframe
  as.data.frame(.)