根据规则删除多列和多行中的重复项

Question

假设我有以下数据：

dt <- data.frame(id=c(1,1,2,2,3,3,3,4,5,5,5,5,6,7,7),
             rk=c("a","a","b","b","c","y","c","d","e","y","e","e","f","g","h"),
             .id=c("df1", "df9", "df5", "df16", "df2", "df11", "df11", "df4", "df9", "df4", "df6", "df3", "df16", "df2", "df9"))

所以我的数据看起来像这样：

id   rk  .id
1    a   df1
1    a   df9
2    b   df5
2    b  df16
3    c   df2
3    y  df11
3    c  df11
4    d   df4
5    e   df9
5    y   df4
5    e   df6
5    e   df3
6    f  df16
7    g   df2
7    h   df9

但我只希望每对 id 和 rk 有 一个 行。 因此，在示例中，id=5 可以有两行：一行 rk=e，另一行 rk=y。

要找到要保留的正确行，我查看 .id 列。这里我按以下顺序排列优先级：

df2、df9、df1、df5、df4、df6、df15、df17、df16、df14、df8、df11、df3、df7、df12、df13、df10

所以我总是在 .id=df9 的一行上保留 .id=df2 的一行。同样，我将始终保留 .id=df15 的行而不是 .id=df14 的行。

请注意，顺序不是按时间顺序排列的。

回到我的示例数据，这就是我想要得到的结果：

id   rk  .id
1    a   df9
2    b   df5
3    c   df2
3    y  df11
4    d   df4
5    e   df9
5    y   df4
6    f  df16
7    g   df2
7    h   df9

我的数据集很大，所以我希望你们中的一些人可以帮助我编写一些代码，让这很容易。

Answer 1

使用dplyr，我们可以通过group_by id 和rk 获得.id 的第一个match 和new_order。

library(dplyr)
dt %>%
  group_by(id, rk) %>%
  summarise(.id = .id[which.min(match(.id, new_order))])

#   id rk    .id  
#   <dbl> <fct> <fct>
# 1  1.00 a     df9  
# 2  2.00 b     df5  
# 3  3.00 c     df2  
# 4  3.00 y     df11 
# 5  4.00 d     df4  
# 6  5.00 e     df9  
# 7  5.00 y     df4  
# 8  6.00 f     df16 
# 9  7.00 g     df2  
#10  7.00 h     df9 

---

等价，base R aggregate 选项是

aggregate(.id~id+rk, dt, function(x) x[which.min(match(x, new_order))]) 

---

如果我们还想保留一些其他的列，我们可以使用filter 而不是summarise

dt %>%
 group_by(id, rk) %>%
 filter(.id == .id[which.min(match(.id, new_order))])

---

其等效的ave 选项将是

dt[with(dt, .id ==  ave(.id, id, rk, FUN = function(x) 
                    x[which.min(match(x, new_order))])), ]

在哪里，

new_order <- c("df2", "df9", "df1", "df5", "df4", "df6", "df15", "df17", "df16",
           "df14", "df6", "df8", "df11", "df3", "df7", "df12", "df13", "df10")

Answer 2

我会像这样使用data.table。看起来略长但相当直观。

library(data.table)

# Load datasets
dt <- data.frame(id=c(1,1,2,2,3,3,3,4,5,5,5,5,6,7,7),
                 rk=c("a","a","b","b","c","y","c","d","e","y","e","e","f","g","h"),
                 .id=c("df1", "df9", "df5", "df16", "df2", "df11", "df11", "df4", "df9", "df4", "df6", "df3", "df16", "df2", "df9"))


Priority_List <- c("df2", "df9", "df1", "df5", "df4", "df6", "df15", "df17", "df16",
                   "df14", "df6", "df8", "df11", "df3", "df7", "df12", "df13", "df10")

# Create a data table called priority list with the priority rank
Priority_List <- data.table(.id = Priority_List , Priority = 1:length(Priority_List))

# Convert your parent data.frame into data.table
dt <- data.table(dt)

# Merge the Priority List with dt based on .id
dt <- merge(dt,Priority_List, by =c(".id"), all.x = TRUE)

# Find the minimum priority for each id and rk
dt <- dt[, Min_Priority := min(Priority), by = c("id", "rk")]

# Filter when Priority is equal to the Min_Priority for a particular id, rk
dt <- dt[Min_Priority == Priority]

# Take unique in case there are duplicate rows.
dt <- unique(dt)

# Remove unwanted columns and order based on id and rk
dt <- dt[,.(id, rk, .id)][order(id, rk)]

希望这会有所帮助。