匹配两个data.tables的行以填充data.table的子集[重复]

Question

我有两个数据表：

> DT1 <- data.table(col1 = c("a","b","b","a","c","b","a","c")
                  , col2 = c("b","d","c","a","d","a","c","a")
                  , col3 = c(1,2,3,4,5,6,7,8))
> DT2 <- data.table(col1 = c("b","e","c","e","b","c","d","a")
                  , col2 = c("d","b","c","d","a","a","c","a")
                  , col3 = c(NA,1,2,NA,6,NA,3,NA))

> DT1
   col1 col2 col3
1:    a    b    1
2:    b    d    2
3:    b    c    3
4:    a    a    4
5:    c    d    5
6:    b    a    6
7:    a    c    7
8:    c    a    8

> DT2
   col1 col2 col3
1:    b    d   NA
2:    e    b    1
3:    c    c    2
4:    e    d   NA
5:    b    a    6
6:    c    a   NA
7:    d    c    3
8:    a    a   NA

我想使用 col1 和 col2 将 DT2 的 col3 为 NA 的行与 DT1 的行进行匹配，如果匹配存在，则用 DT1 中的值填充 DT2 中 col3 的 NA 值。

> #desired Output
> DT2_output
   col1 col2 col3
1:    b    d    2
2:    e    b    1
3:    c    c    2
4:    e    d   NA
5:    b    a    6
6:    c    a    8
7:    d    c    3
8:    a    a    4

如何使用简洁的 data.table 操作（无循环）来执行此操作，因为每个 data.table 中有数百万行。 我尝试了以下操作，但它给了我错误，我认为这与 which 语句有关。

> ##doesn't work
> DT2[is.na(col3), col3 := DT1[which(col1 == DT2[is.na(col3),col1] && col2 == DT2[is.na(col3), col2]), col3]]

Answer 1

我可以直接做一个左连接，然后根据原始 col3 是否丢失的 ifelse 条件，如下所示：

DT2new <- merge(DT2, DT1, by = c("col1", "col2"), all.x = T)
DT2new[, col3 := ifelse(is.na(col3.x), col3.y, col3.x)]
DT2new <- DT2new[, .(col1, col2, col3)]

#   col1 col2 col3
#1:    a    a    4
#2:    b    a    6
#3:    b    d    2
#4:    c    a    8
#5:    c    c    2
#6:    d    c    3
#7:    e    b    1
#8:    e    d   NA

另外，一种更有效的方法是通过引用进行操作，它直接修改（通过引用就地）DT2 data.table：

DT2[DT1, on = .(col1, col2), col3 := i.col3]

#   col1 col2 col3
#1:    b    d    2
#2:    e    b    1
#3:    c    c    2
#4:    e    d   NA
#5:    b    a    6
#6:    c    a    8
#7:    d    c    3
#8:    a    a    4

i.col3 中的i 指的是DT[i, j, by] 中的i，因此指的是DT1 中的col3 列。这是有效的，因为您没有任何情况会发生两个 data.tables 中列 col1 和 col2 的非缺失匹配。如果您有这种情况，您可以执行如下更通用的操作（包括 data.tables DT1 和 DT2 的示例）：

DT1 <- data.table(col1 = c("a","b","b","a","c","b","a","c", "e"), 
                  col2 = c("b","d","c","a","d","a","c","a", "b"),
                  col3 = c(1,2,3,4,5,6,7,8, 22))
DT2 <- data.table(col1 = c("b","e","c","e","b","c","d","a"),
                  col2 = c("d","b","c","d","a","a","c","a"),
                  col3 = c(NA,1,2,NA,6,NA,3,NA))

您可以看到DT1 的col1 = "e" 和col2 = "b" 的值为22。DT2 的col1 = "e" 和col2 = "b" 的值为1。为了优先考虑@987654338 @如果发生这种冲突，你会这样做：

DT2[DT1, on = .(col1, col2), col3 := ifelse(is.na(x.col3), i.col3, x.col3)]

这给了你

#   col1 col2 col3
#1:    b    d    2
#2:    e    b    1
#3:    c    c    2
#4:    e    d   NA
#5:    b    a    6
#6:    c    a    8
#7:    d    c    3
#8:    a    a    4

x.col3 中的x. 指的是DT2 中的col3。

编辑：一种更有效的通用方法是矢量化 data.table 操作方法（如果 DT1 包含特定对 col1 和 col2 的值）

鉴于有用的 cmets (@Frank & @chinsoon12)，我仔细检查了提供的解决方案。正如所指出的，ifelse 可能会变慢（原因在评论中），这就是为什么矢量化解决方案是更好的方法：

dt1 <- data.table(col1 = paste0(rep(letters[1:26], times = 10000), rep(c(1:10000), each = 26)),
                  col2 = paste0(rep(letters[26:1], times = 10000), rep(c(1:10000), each = 26)),
                  col3 = rep(c(3,1,2,4,6,5,3,6,2,NA), times = 26000))

dt2 <- data.table(col1 = paste0(rep(letters[1:26], times = 10000), rep(c(1:10000), each = 26)),
                  col2 = paste0(rep(letters[26:1], times = 10000), rep(c(1:10000), each = 26)),
                  col3 = rep(c(10,NA,22,51,60,NA,32,NA,28,NA), times = 26000))
dt2alt <- copy(dt2)

microbenchmark::microbenchmark(
NoIfelse = dt2[is.na(col3), col3 := dt1[dt2[is.na(col3)], on = .(col1, col2), col3]],
Ifelse = dt2alt[dt1, on = .(col1, col2), col3 := ifelse(is.na(x.col3), i.col3, x.col3)]
)

#Unit: milliseconds
#     expr       min       lq     mean   median       uq      max neval cld
# NoIfelse  94.06635 100.2841 107.4140 103.7807 107.4006 154.7868   100  a 
#   Ifelse 219.94923 243.2651 254.3271 250.6967 266.8300 306.7213   100   b

identical(dt2, dt2alt) #TRUE