基于检查的循环更高效

Question

我编写了一个 for 循环，它会进行一些检查并根据结果返回 0 或 1。但是，在大型数据集上运行它需要很长时间（过夜，早上仍然运行）。关于如何使用dplyr 或其他工具提高效率的任何想法？谢谢

这是一些测试数据：

tdata <- structure(list(cusip = c(2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 
2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 
2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 
2, 2, 2, 2, 2, 2), fyear = c("1971", "1971", "1971", "1971", 
"1971", "1971", "1971", "1971", "1971", "1971", "1971", "1971", 
"1972", "1972", "1972", "1972", "1972", "1972", "1972", "1972", 
"1972", "1972", "1972", "1972", "1972", "1973", "1973", "1973", 
"1973", "1973", "1973", "1973", "1973", "1973", "1973", "1973", 
"1973", "1974", "1974", "1974", "1974", "1974", "1974", "1974", 
"1974", "1974", "1974", "1974", "1974", "1975", "1975", "1975", 
"1975", "1975", "1975", "1975", "1975", "1975", "1975", "1975"
), datadate = c(19711231L, 19710129L, 19710226L, 19710331L, 19710430L, 
19710528L, 19710630L, 19710730L, 19710831L, 19710930L, 19711029L, 
19711130L, 19721231L, 19720131L, 19720229L, 19720330L, 19720428L, 
19720531L, 19720630L, 19720731L, 19720831L, 19720929L, 19721031L, 
19721130L, 19721229L, 19731231L, 19730131L, 19730228L, 19730330L, 
19730430L, 19730531L, 19730629L, 19730731L, 19730831L, 19730928L, 
19731031L, 19731130L, 19741231L, 19740131L, 19740228L, 19740329L, 
19740430L, 19740531L, 19740628L, 19740731L, 19740830L, 19740930L, 
19741031L, 19741129L, 19751231L, 19750131L, 19750228L, 19750331L, 
19750430L, 19750530L, 19750630L, 19750731L, 19750829L, 19750930L, 
19751031L), month = c("12", "01", "02", "03", "04", "05", "06", 
"07", "08", "09", "10", "11", "12", "01", "02", "03", "04", "05", 
"06", "07", "08", "09", "10", "11", "12", "12", "01", "02", "03", 
"04", "05", "06", "07", "08", "09", "10", "11", "12", "01", "02", 
"03", "04", "05", "06", "07", "08", "09", "10", "11", "12", "01", 
"02", "03", "04", "05", "06", "07", "08", "09", "10")), .Names = c("cusip", 
"fyear", "datadate", "month"), row.names = c(NA, -60L), class = c("tbl_df", 
"tbl", "data.frame"))

for循环：

for(i in min(tdata$cusip):max(tdata$cusip)){ 
    for (j in min(tdata$fyear):max(tdata$fyear)) {
      monthcheck <- filter(tdata, cusip == i & (fyear == j-1 | fyear == j-2 | fyear == j-3 | fyear == j-4))
      if((length(monthcheck$month) / 60) >= 0.4) tdata$check[tdata$cusip == i & tdata$fyear ==  j] <- 1
}}

这将返回 1 表示 1973-1975，因为检查通过了。有没有办法让这个for 循环更高效，因为这需要一段时间才能在大型数据集上运行？

编辑：for循环的解释

对于每个唯一 ID (cusip) 和每年 (fyear)，使用 select 获取前 4 年的数据，然后计算观察次数并检查它是否大于 40%。如果是这样，请为特定的 cusip 分配 1 到 tdata$check。

我们的想法是确保每个唯一 id 至少有 60 个前一个月观察值中的 24 个。

Answer 1

分组和滞后累积和的解决方案：

library(dplyr)

tdata %>%
  group_by(cusip, fyear) %>%
  summarise(number = n(), share = n() / 60)  %>% 
  mutate( cum_y = lag(cumsum(share)), 
          cum_y4 = lag(cum_y, 4),
          last4 = ifelse(is.na(cum_y4), cum_y, cum_y - cum_y4),
          check = as.numeric( last4 >= 0.4 )
          ) %>%
  select(cusip, fyear, last4, check)

解释：

1. 按fyear 分组，统计观察并获得share 一年
2. cum_y 是滞后的累计份额总和
3. cum_y4 落后了 4 年 cum_y
4. last4 是 cum_y 和 cum_y4 之间的区别
5. check 正在检查 last4

更新

加入原始数据中的变量：

... %>%
  left_join(tdata, by = c("cusip", "fyear"))