在 R 中查找两个数据帧之间的重叠，如何使我的代码更高效？

Question

我在 R 中有两个数据框。在第一个中，我有两列，一列称为“chr”，另一列称为“位置”；在第二个数据框中，我有三列，一列又是“chr”，另一列是“开始”，另一列是“结束”。我想选择第一个数据帧中 chr 值与第二个数据帧相同的行，但其“位置”位于第二个数据帧的间隔开始端。

为此，我在 R 中编写了一个函数，它可以为我提供所需的输出，但是当我使用大量数据帧运行它时它非常慢。

# My DataFrames are:

bed <- data.frame(Chr = c(rep("chr1",4),rep("chr2",3),rep("chr3",1)),
                  x1 = c(5,20,44,67,5,20,44,20),
                  x3=c(12,43,64,94,12,43,64,63))

snv <- data.frame(Chr = c(rep("chr1",6),rep("chr3",6)),
                  position = c(5,18,46,60,80,90,21,60,75,80,84,87))

# My function is:

get_overlap <- function(df, position, chrom){
  overlap <- FALSE
  for (row in 1:nrow(df)){
    chr = df[row, 1]
    start = df[row, 2]
    end = df[row, 3]
    if(chr == chrom & position %in% seq(start, end)){
      overlap <- TRUE
    }
    }
  return(overlap)
}

# The code is:

overlap_vector = c()
for (row in 1:nrow(snv)){
  chrom = snv[row, 1]
  position = snv[row, 2]
  overlap <- get_overlap(bed, position, chrom)
  overlap_vector <- c(overlap_vector, overlap)
}

print(snv[overlap_vector,])

如何提高效率？我从未使用过哈希表，这可以是答案吗？

Answer 1

我确信有一个更优雅的data.table 解决方案，但它有效。首先我加载包。

# Load package
library(data.table)

然后，我定义数据表

# Define data tables
bed <- data.table(Chr = c(rep("chr1",4),rep("chr2",3),rep("chr3",1)),
                  start = c(5,20,44,67,5,20,44,20),
                  end = c(12,43,64,94,12,43,64,63))

snv <- data.table(Chr = c(rep("chr1",6),rep("chr3",6)),
                  position = c(5,18,46,60,80,90,21,60,75,80,84,87))

在这里，我对position 和start/end 进行非等连接，并在Chr 上进行等连接。我假设您想保留所有列，因此在 j 参数中指定它们并省略那些不匹配的行。

na.omit(bed[snv, 
            .(Chr, start = x.start, end = x.end, position = i.position), 
            on = c("start <= position", "end >= position", "Chr == Chr")])
#>     Chr start end position
#> 1: chr1     5  12        5
#> 2: chr1    44  64       46
#> 3: chr1    44  64       60
#> 4: chr1    67  94       80
#> 5: chr1    67  94       90
#> 6: chr3    20  63       21
#> 7: chr3    20  63       60

^{由reprex package (v0.3.0) 于 2019 年 8 月 21 日创建}

---

编辑

快速基准测试表明，Nathan 的解决方案大约是原来的两倍！

Unit: milliseconds
         expr      min       lq     mean   median       uq      max neval
 NathanWren() 1.684392 1.729557 1.819263 1.751520 1.787829 5.138546   100
   Lyngbakr() 3.336902 3.395528 3.603376 3.441933 3.496131 7.720925   100

Answer 2

data.table 包非常适合快速合并表格。它还带有一个矢量化的between 函数，仅用于此类任务。

library(data.table)

# Convert the data.frames to data.tables
setDT(bed)
setDT(snv)

# Use the join syntax for data.table, then filter for the desired rows
overlap_dt <- bed[
  snv,
  on = "Chr",
  allow.cartesian = TRUE # many-to-many matching
][
  between(position, lower = x1, upper = x3)
]

overlap_dt
#     Chr x1 x3 position
# 1: chr1  5 12        5
# 2: chr1 44 64       46
# 3: chr1 44 64       60
# 4: chr1 67 94       80
# 5: chr1 67 94       90
# 6: chr3 20 63       21
# 7: chr3 20 63       60