data.table 分组操作，列的变量名称没有慢 DT[, mean(get(column Name)), by = grp]

Question

我想创建一个使用列变量名和数据变量名的函数。

这个功能是我想要的，它可以工作：

n <- 1e7
d <- data.table(x = 1:n, grp = sample(1:1e5, n, replace = T))
dataName = "d"
colName = "x"

# Objective :
FOO <- function(dataName = "d",
         colName = "x"){
  get(dataName)[, mean(get(colName)), by = grp]
}

问题是对每个组的get() 进行评估非常耗时。在真实数据示例中，它比等效的静态名称长 14 倍。我想达到与列名是静态的相同的执行时间。

我尝试了什么：

(cl <- substitute(mean(eval(parse(text = colName))), list(colName = as.name(colName))))

microbenchmark::microbenchmark(

  # 1) works and quick but does not use variable names of columns (654ms)
  (t1 <- d[, mean(x), by = grp]),

  # 2) works but slow (1006ms)
  (t2 <- get(dataName)[, mean(get(colName)), by = grp]), # works but slow

  # 3) works but slow (4075ms)
  (t3 <- eval(parse(text = dataName))[, mean(eval(parse(text = colName))), by = grp]),

  # 4) works but very slow (37202ms)
  (t4 <- get(dataName)[, eval(cl), by = grp]),

  # 5) double dot syntax doesn't work cause I don't master it
  # (t5 <- get(dataName)[, mean(..colName), by = grp]),

  times = 10)

双点语法在这里合适吗？为什么 4) 这么慢？我从this post 那里得到它，这是最好的选择。我从 post 改编了双点语法。

非常感谢您的帮助！

Answer 1

最好将数据集名称d 传递给FOO 函数，而不是传递字符串"d"。此外，您可以将lapply 与.SD 结合使用，这样您就可以从内部优化中受益，而不是使用mean(get(colName))。

FOO2 = function(dataName=d, colName = "x") { # d instead of "d" passed to the first argument!
  dataName[, lapply(.SD, mean), by=grp, .SDcols=colName]
}

基准测试：FOO 与 FOO2

set.seed(147852)
n <- 1e7
d <- data.table(x = 1:n, grp = sample(1:1e5, n, replace = T))

microbenchmark::microbenchmark(
  FOO(),
  FOO2(),
  times=5L
)

Unit: milliseconds
   expr       min        lq      mean    median        uq       max neval
  FOO() 4632.4014 4672.7781 4787.4958 4707.9023 4846.7081 5077.6893     5
 FOO2()  255.0828  267.1322  297.0389  275.4467  281.9873  405.5456     5