获取一段时间内的累积值计数答案

【问题标题】：Get a count of cumulative values over time获取一段时间内的累积值计数
【发布时间】：2021-11-09 04:17:34
【问题描述】：

假设我有以下数据。

dt = data.table(
  date = c("2020-10-01", "2020-10-02", "2020-10-03", "2020-10-01", "2020-10-01",
           "2020-10-03", "2020-10-04", "2020-10-04", "2020-10-05", "2020-10-05"),
  client = sample(LETTERS[1:3], 10, replace = TRUE),
  vals = rnorm(10))

dt[order(date)]

dt2 = dt[order(date), .(sum_vals = sum(vals)), by = .(date, client)]

         date client    sum_vals
1: 2020-10-01      B  2.53737527
2: 2020-10-01      C  0.64366866
3: 2020-10-02      A  1.01776243
4: 2020-10-03      C -0.06303562
5: 2020-10-03      A  0.63702089
6: 2020-10-04      B  0.12681052
7: 2020-10-04      A  0.82889616
8: 2020-10-05      B -1.45734539
9: 2020-10-05      C  0.02594185

我想做的是按日期计算累积客户数。

所以在这种情况下，它看起来像这样。

         date.  acts 
1: 2020-10-01      2   # b and c we active on 10/01 or before
2: 2020-10-02      3   # a, b and c we active on 10/02 or before
3: 2020-10-03      3   # a, b and c we active on 10/03 or before
4: 2020-10-04      3   # a, b and c we active on 10/04 or before
5: 2020-10-05      3   # a, b and c we active on 10/05 or before

关于如何使用 data.table 或 dplyr 实现这一点的任何想法？

【问题讨论】：

您是从dt 还是dt2 开始以获得您想要的结果？
从 dt2 开始，因为 dt 中的一些值需要相加
对于未来的问题，请 (a) 使用 set.seed，或 (b) 使用 dput(.) 向我们提供数据。否则，答案不太可能与您从同一点开始（无需手动抓取您的数据）。

标签： r dplyr data.table

【解决方案1】：

我们可以做到

dt2[,  .(date = unique(date), acts = unlist(lapply(unique(date),  
         function(x) uniqueN(client[date <= x]))))]

-输出

          date acts
1: 2020-10-01    2
2: 2020-10-02    2
3: 2020-10-03    3
4: 2020-10-04    3
5: 2020-10-05    3

【讨论】：

【解决方案2】：

这是另一种选择：

dt2[, .(date, v=cumsum(!duplicated(client)))][, .(acts=max(v)), date]

输出：

         date acts
1: 2020-10-01    2
2: 2020-10-02    3
3: 2020-10-03    3
4: 2020-10-04    3
5: 2020-10-05    3

数据：

library(data.table)
dt2 = fread("date client    sum_vals
2020-10-01      B  2.53737527
2020-10-01      C  0.64366866
2020-10-02      A  1.01776243
2020-10-03      C -0.06303562
2020-10-03      A  0.63702089
2020-10-04      B  0.12681052
2020-10-04      A  0.82889616
2020-10-05      B -1.45734539
2020-10-05      C  0.02594185")

编辑：更大数据集的采样时间

数据（约 1mio 行，5 年内有 10k 个客户端）和时序代码：

library(data.table)
set.seed(0L)
nd <- 5L * 365L
nr <- 1e6L
nc <- 1e4L
dt2 <- data.table(date=sample(nd, nr, TRUE), client=sample(nc, nr, TRUE), vals=1L)[,
    .(vals=sum(vals)), keyby=.(date, client)]

microbenchmark::microbenchmark(times=1L,
    m0={a0 <- dt2[,  .(date = unique(date), acts = unlist(lapply(unique(date),  
         function(x) uniqueN(client[date <= x]))))]},
    m1={a1 <- dt2[, .(date, v=cumsum(!duplicated(client)))][, .(acts=max(v)), date]},
    m2={a2 <- dt2[, .(clients = list(unique(client))), by = date
        ][, acts := lengths(Reduce(function(p,n) unique(c(p,n)), clients, accumulate = TRUE))]}
) #in chronological order of post
fsetequal(a0, a1)
#[1] TRUE
fsetequal(a0, a2[, clients:=NULL])
#[1] TRUE

时间安排：

Unit: milliseconds
 expr        min         lq       mean     median         uq        max neval
   m0 14293.6303 14293.6303 14293.6303 14293.6303 14293.6303 14293.6303     1
   m1    40.2999    40.2999    40.2999    40.2999    40.2999    40.2999     1
   m2  1088.7080  1088.7080  1088.7080  1088.7080  1088.7080  1088.7080     1

【讨论】：

【解决方案3】：

另一种选择：

dt2[, .(clients = list(unique(client))), by = date
  ][, accts := lengths(Reduce(function(p,n) unique(c(p,n)), clients, accumulate = TRUE))]
#          date clients accts
#        <IDat>  <list> <int>
# 1: 2020-10-01     B,C     2
# 2: 2020-10-02       A     3
# 3: 2020-10-03     C,A     3
# 4: 2020-10-04     B,A     3
# 5: 2020-10-05     B,C     3

数据

dt2 <- setDT(structure(list(date = structure(c(18536L, 18536L, 18537L, 18538L, 18538L, 18539L, 18539L, 18540L, 18540L), class = c("IDate", "Date")), client = c("B", "C", "A", "C", "A", "B", "A", "B", "C"), sum_vals = c(2.53737527, 0.64366866, 1.01776243, -0.06303562, 0.63702089, 0.12681052, 0.82889616, -1.45734539, 0.02594185)), row.names = c(NA, -9L), class = c("data.table", "data.frame")))

基准测试，

bench::mark(
  chinsoon12 = dt2[, .(date, v=cumsum(!duplicated(client)))][, .(acts=max(v)), date], 
  akrun = dt2[, .(date = unique(date), acts = unlist(lapply(unique(date), function(x) uniqueN(client[date <= x]))))], 
  r2evans = dt2[, .(clients = list(unique(client))), by = date][, accts := lengths(Reduce(function(p,n) unique(c(p,n)), clients, accumulate = TRUE))], 
  check = FALSE)
# # A tibble: 3 x 13
#   expression      min   median `itr/sec` mem_alloc `gc/sec` n_itr  n_gc total_time result memory        time     gc        
#   <bch:expr> <bch:tm> <bch:tm>     <dbl> <bch:byt>    <dbl> <int> <dbl>   <bch:tm> <list> <list>        <list>   <list>    
# 1 chinsoon12   2.17ms   2.61ms      359.    97.7KB     0      180     0      502ms <NULL> <Rprofmem[,3~ <bch:tm~ <tibble [~
# 2 akrun       810.3us  923.5us     1010.    48.7KB     0      505     0      500ms <NULL> <Rprofmem[,3~ <bch:tm~ <tibble [~
# 3 r2evans     843.7us  956.5us      913.    81.1KB     2.45   373     1      409ms <NULL> <Rprofmem[,3~ <bch:tm~ <tibble [~

【讨论】：

根据microbenchmark 几乎快两倍：赞成！
请注意，@akrun 结果似乎与您的不同：2 第一行是 2 而不是 2,3
我看到了。我认为这是因为 akrun 可能从随机的 dt 开始，而不是从控制台输出中抓取。
@akrun，我认为您的解决方案是最快的有什么问题？
@chinsoon12 它是dt2，如答案中所提供。我还尝试了 1000 或 10000，发现 order 没有变化。但诚然，这有点仓促，如果 OP 真的关心性能，那么更明智的基准是合理的。