【问题标题】:How to calculate means for increasing time windows in R如何计算R中增加时间窗的方法
【发布时间】:2019-04-26 21:10:09
【问题描述】:

我有一个数据框,其中包含许多不同的动物(以下示例数据中的 a、b、c)、transactionID、计数和天数。我想计算每个动物内每个 transactionID 的增加时间窗口(按天指定)的计数值的平均值和标准偏差。即对于动物a的事务ID 1,我想为i)天-1到-2,ii)天-1到-3,iii)天-1到-4等的平均值和标准差添加列......所以我最终得到了 5 个新列,增加了时间窗口,5 个用于 SD。

示例数据:

> dput(df)
structure(list(Animal = c("a", "a", "a", "a", "a", "a", "a", "a", 
"a", "a", "a", "b", "b", "b", "b", "b", "b", "b", "c", "c", "c", 
"c", "c"), Count = c(45L, 54L, 22L, 3L, 23L, 46L, 45L, 22L, 67L, 
34L, 22L, 34L, 677L, 86L, 54L, 4L, 56L, 98L, 23L, 54L, 22L, 77L, 
23L), Day = c(-6L, -5L, -4L, -3L, -2L, -1L, -5L, -4L, -3L, -2L, 
-1L, -4L, -3L, -2L, -1L, -3L, -2L, -1L, -6L, -5L, -3L, -2L, -1L
), transactionID = c(1L, 1L, 1L, 1L, 1L, 1L, 2L, 2L, 2L, 2L, 
2L, 3L, 3L, 3L, 3L, 4L, 4L, 4L, 5L, 5L, 5L, 5L, 5L)), .Names = c("Animal", 
"Count", "Day", "transactionID"), class = "data.frame", row.names = c(NA, 
-23L))

> df
   Animal Count Day transactionID
1     a    45  -6             1
2     a    54  -5             1
3     a    22  -4             1
4     a     3  -3             1
5     a    23  -2             1
6     a    46  -1             1
7     a    45  -5             2
8     a    22  -4             2
9     a    67  -3             2
10    a    34  -2             2
11    a    22  -1             2
12    b    34  -4             3
13    b   677  -3             3
14    b    86  -2             3
15    b    54  -1             3
16    b     4  -3             4
17    b    56  -2             4
18    b    98  -1             4
19    c    23  -6             5
20    c    54  -5             5
21    c    22  -3             5
22    c    77  -2             5
23    c    23  -1             5

我可以使用下面的代码实现我想要的输出。但是,当循环遍历我的整个数据框时,有些情况下我的动物时间少于 6 天,并且在最大编号的情况下,此 for 循环不会添加 NA。数据集中的天数(因此时间窗口)小于 6。我的数据集中还有一些案例,其中日列中缺少一天(即动物 c,第 -4 天)。在这种情况下,我想从丢失的那一天开始为所有时间窗口添加均值和 sds 的 NA。请参阅下面的我想要的输出。

我的尝试:

#create empty matrix
res2 = as.data.frame(matrix(NA,0,14))
#split by name
animal.list = split(df,df$Name)

#For loop for 
for(i in 1:length(animal.list)){
  a = as.data.frame(animal.list[[i]])
  animal = unique(a$Name)
  #create empty matrix
  res = as.data.frame(matrix(NA,0,14))

  #create list of event IDs
  event = split(a,a$transactionID)

  #loop through each event in turn and calculate the mean of different baseline periods (from 2 days to 6 days)- clunky!
  for(j in 1:length(event)){
    e = as.data.frame(event[[j]])
    #max day
    e$maxday = unique(e[1,]$Day)
    #save mean activity value for the 2 days prior to event
    e$mean2d = round(mean(e[e$Day >-3,]$Count),3)
    e$SD2d = round(sd(e[e$Day >-3,]$Count),3)
    #save mean activity value for the 3 days prior to event
    e$mean3d = round(mean(e[e$Day >-4,]$Count),3)
    e$SD3d = round(sd(e[e$Day >-4,]$Count),3)
    #save mean activity value for the 4 days prior to event
    e$mean4d = round(mean(e[e$Day >-5,]$Count),3)
    e$SD4d = round(sd(e[e$Day >-5,]$Count),3)
    #save mean activity value for the 5 days prior to event
    e$mean5d = round(mean(e[e$Day >-6,]$Count),3)
    e$SD5d = round(sd(e[e$Day >-6,]$Count),3)
    #save mean activity value for the 6 days prior to event
    e$mean6d = round(mean(e[e$Day >-7,]$Count),3)
    e$SD6d = round(sd(e[e$Day >-7,]$Count),3)
    res = rbind(res,e)
  }
  res2 = rbind(res2,res) 
}

期望的输出:

>res2
   Name Count Day transactionID maxday mean2d   SD2d  mean3d    SD3d mean4d    SD4d mean5d   SD5d
1     a    45  -6             1     -6   34.5 16.263  24.000  21.517  23.50  17.597   29.6 20.452
2     a    54  -5             1     -6   34.5 16.263  24.000  21.517  23.50  17.597   29.6 20.452
3     a    22  -4             1     -6   34.5 16.263  24.000  21.517  23.50  17.597   29.6 20.452
4     a     3  -3             1     -6   34.5 16.263  24.000  21.517  23.50  17.597   29.6 20.452
5     a    23  -2             1     -6   34.5 16.263  24.000  21.517  23.50  17.597   29.6 20.452
6     a    46  -1             1     -6   34.5 16.263  24.000  21.517  23.50  17.597   29.6 20.452
7     a    45  -5             2     -5   28.0  8.485  41.000  23.302  36.25  21.266   38.0 18.828
8     a    22  -4             2     -5   28.0  8.485  41.000  23.302  36.25  21.266   38.0 18.828
9     a    67  -3             2     -5   28.0  8.485  41.000  23.302  36.25  21.266   38.0 18.828
10    a    34  -2             2     -5   28.0  8.485  41.000  23.302  36.25  21.266   38.0 18.828
11    a    22  -1             2     -5   28.0  8.485  41.000  23.302  36.25  21.266   38.0 18.828
12    b    34  -4             3     -4   70.0 22.627 272.333 350.817 212.75 310.240     NA     NA
13    b   677  -3             3     -4   70.0 22.627 272.333 350.817 212.75 310.240     NA     NA
14    b    86  -2             3     -4   70.0 22.627 272.333 350.817 212.75 310.240     NA     NA
15    b    54  -1             3     -4   70.0 22.627 272.333 350.817 212.75 310.240     NA     NA
16    b     4  -3             4     -3   77.0 29.698  52.667  47.089     NA      NA     NA     NA
17    b    56  -2             4     -3   77.0 29.698  52.667  47.089     NA      NA     NA     NA
18    b    98  -1             4     -3   77.0 29.698  52.667  47.089     NA      NA     NA     NA
19    c    23  -6             5     -6   50.0 38.184      NA      NA     NA      NA     NA     NA
20    c    54  -5             5     -6   50.0 38.184      NA      NA     NA      NA     NA     NA
21    c    22  -3             5     -6   50.0 38.184      NA      NA     NA      NA     NA     NA
22    c    77  -2             5     -6   50.0 38.184      NA      NA     NA      NA     NA     NA
23    c    23  -1             5     -6   50.0 38.184      NA      NA     NA      NA     NA     NA
   mean6d   SD6d
1  32.167 19.343
2  32.167 19.343
3  32.167 19.343
4  32.167 19.343
5  32.167 19.343
6  32.167 19.343
7      NA     NA
8      NA     NA
9      NA     NA
10     NA     NA
11     NA     NA
12     NA     NA
13     NA     NA
14     NA     NA
15     NA     NA
16     NA     NA
17     NA     NA
18     NA     NA
19     NA     NA
20     NA     NA
21     NA     NA
22     NA     NA
23     NA     NA

编辑:基于@Henrik 的建议(这是一种更快的方法来计算累积均值和广告,但仍然没有考虑缺失的情况天并在这些情况下使用 Has)- 任何简单的建议将不胜感激):

library(dplyr)
library(TTR)

#create empty matrix
res2 = as.data.frame(matrix(NA,0,14))
#split by name
animal.list = split(df,df$Name)

#For loop for 
for(i in 1:length(animal.list)){
  a = as.data.frame(animal.list[[i]])
  animal = unique(a$Name)
  #create empty matrix
  res = as.data.frame(matrix(NA,0,14))
  #create list of event IDs
  event = split(a,a$transactionID)

  #loop through each event in turn and calculate the mean of different baseline periods (from 2 days prior to 10 days prior)
  for(j in 1:length(event)){
    e = as.data.frame(event[[j]])
    #max day
    e$maxday = unique(e[1,]$Day)
    cmean = cummean(rev(e$Count))
    csd= runSD(rev(e$Count),n=1,cumulative=TRUE)
    e$mean2d = cmean[2]
    e$sd2d = csd[2]
    e$mean3d = cmean[3]
    e$sd3d = csd[3]
    e$mean4d = cmean[4]
    e$sd4d = csd[4]
    e$mean5d = cmean[5]
    e$sd5d = csd[5]
    e$mean6d = cmean[6]
    e$sd6d = csd[6]
    res = rbind(res,e)
  }
  res2 = rbind(res2,res) 
}

【问题讨论】:

  • @Henrik - 谢谢你的这些建议。非常有帮助,这给了我想要的一切输出,除了考虑缺少一天的情况。您能否就如何将其合并到上面的 forloop 编辑中提出建议?任何帮助将不胜感激。谢谢!
  • @jjulip 你看过我的回答了吗?它与您想要的输出完全匹配,而且更简单。
  • @arg0naut 是的,非常感谢。我一直在努力解决它。我只是无法通过代码,因为它对我来说太先进了。即请在下面查看我的问题。我真的很想提高我的 R 技能,但是很难用大量的 ifelses 分解大功能,而且我需要时间!谢谢你的帮助。
  • 警告:切勿在循环中使用rbind!这会导致过度复制!见R Inferno: Growing Objects

标签: r for-loop mean


【解决方案1】:

一个方便的data.table / sapply 函数(为了更好的可读性分为两部分):

add_mean_sd <- function(df, group_var = c("Animal", "transactionID"), day_var = "Day", count_var = "Count", window_var = 2) {

  require(data.table)

  # Calculate mean for the desired window

  df <- setDT(df)[, paste("mean", window_var, "d", sep = "") := ifelse(last(sapply(get(day_var), function(x) length(get(count_var)[between(get(day_var), x - window_var, x)]))) < window_var |
                                                                         any(sapply(get(day_var), function(x) !all(abs(diff(get(day_var)[between(get(day_var), x - window_var + 1, x)])) == 1))), NA_real_, 
                                                                       last(sapply(get(day_var), function(x) round(mean(get(count_var)[between(get(day_var), x - window_var + 1, x)]),3)))), by = mget(group_var)]

  # Calculate sd for the desired window

  df <- df[, paste("sd", window_var, "d", sep = "") := ifelse(last(sapply(get(day_var), function(x) length(get(count_var)[between(get(day_var), x - window_var, x)]))) < window_var |
                                                                any(sapply(get(day_var), function(x) !all(abs(diff(get(day_var)[between(get(day_var), x - window_var + 1, x)])) == 1))), NA_real_, 
                                                              last(sapply(get(day_var), function(x) round(sd(get(count_var)[between(get(day_var), x - window_var + 1, x)]),3)))), by = mget(group_var)]

  return(df)

}

您可以单独使用,在magrittr 管道中使用,或者如果您有许多所需的窗口,则可以使用简单的循环:

# Alone

df <- add_mean_sd(df) # I've set window 2 as default so no need to specify
df <- add_mean_sd(df, window_var = 3) # etc..

# Magrittr

library(magrittr)

df <- add_mean_sd(df) %>% 
  add_mean_sd(window_var = 3) %>%
  add_mean_sd(window_var = 4) %>%
  add_mean_sd(window_var = 5) %>%
  add_mean_sd(window_var = 6)

# A simple loop (will create columns for all windows at once)

for (i in 2:6) { df <- add_mean_sd(df, window_var = i) }

输出:

df[]

    Animal Count Day transactionID mean2d   sd2d  mean3d    sd3d mean4d    sd4d mean5d   sd5d mean6d   sd6d
 1:      a    45  -6             1   34.5 16.263  24.000  21.517  23.50  17.597   29.6 20.452 32.167 19.343
 2:      a    54  -5             1   34.5 16.263  24.000  21.517  23.50  17.597   29.6 20.452 32.167 19.343
 3:      a    22  -4             1   34.5 16.263  24.000  21.517  23.50  17.597   29.6 20.452 32.167 19.343
 4:      a     3  -3             1   34.5 16.263  24.000  21.517  23.50  17.597   29.6 20.452 32.167 19.343
 5:      a    23  -2             1   34.5 16.263  24.000  21.517  23.50  17.597   29.6 20.452 32.167 19.343
 6:      a    46  -1             1   34.5 16.263  24.000  21.517  23.50  17.597   29.6 20.452 32.167 19.343
 7:      a    45  -5             2   28.0  8.485  41.000  23.302  36.25  21.266   38.0 18.828     NA     NA
 8:      a    22  -4             2   28.0  8.485  41.000  23.302  36.25  21.266   38.0 18.828     NA     NA
 9:      a    67  -3             2   28.0  8.485  41.000  23.302  36.25  21.266   38.0 18.828     NA     NA
10:      a    34  -2             2   28.0  8.485  41.000  23.302  36.25  21.266   38.0 18.828     NA     NA
11:      a    22  -1             2   28.0  8.485  41.000  23.302  36.25  21.266   38.0 18.828     NA     NA
12:      b    34  -4             3   70.0 22.627 272.333 350.817 212.75 310.240     NA     NA     NA     NA
13:      b   677  -3             3   70.0 22.627 272.333 350.817 212.75 310.240     NA     NA     NA     NA
14:      b    86  -2             3   70.0 22.627 272.333 350.817 212.75 310.240     NA     NA     NA     NA
15:      b    54  -1             3   70.0 22.627 272.333 350.817 212.75 310.240     NA     NA     NA     NA
16:      b     4  -3             4   77.0 29.698  52.667  47.089     NA      NA     NA     NA     NA     NA
17:      b    56  -2             4   77.0 29.698  52.667  47.089     NA      NA     NA     NA     NA     NA
18:      b    98  -1             4   77.0 29.698  52.667  47.089     NA      NA     NA     NA     NA     NA
19:      c    23  -6             5   50.0 38.184      NA      NA     NA      NA     NA     NA     NA     NA
20:      c    54  -5             5   50.0 38.184      NA      NA     NA      NA     NA     NA     NA     NA
21:      c    22  -3             5   50.0 38.184      NA      NA     NA      NA     NA     NA     NA     NA
22:      c    77  -2             5   50.0 38.184      NA      NA     NA      NA     NA     NA     NA     NA
23:      c    23  -1             5   50.0 38.184      NA      NA     NA      NA     NA     NA     NA     NA

如果您想更改顺序(例如从第一个到最后一个),您可以将last 更改为first,同时更改between 参数的参数(反转它们并更改符号)。

这意味着您所需大小的窗口的平均值将从第一个值开始按预期取值。

例子:

add_mean_sd <- function(df, group_var = c("Animal", "transactionID"), day_var = "Day", count_var = "Count", window_var = 2) {

  require(data.table)

  # Calculate mean for the desired window

  df <- setDT(df)[, paste("mean", window_var, "d", sep = "") := ifelse(first(sapply(get(day_var), function(x) length(get(count_var)[between(get(day_var), x, x + window_var)]))) < window_var |
                                                                         any(sapply(get(day_var), function(x) !all(abs(diff(get(day_var)[between(get(day_var), x, x + window_var - 1)])) == 1))), NA_real_, 
                                                                       first(sapply(get(day_var), function(x) round(mean(get(count_var)[between(get(day_var), x, x + window_var - 1)]),3)))), by = mget(group_var)]

  # Calculate sd for the desired window

  df <- df[, paste("sd", window_var, "d", sep = "") := ifelse(first(sapply(get(day_var), function(x) length(get(count_var)[between(get(day_var), x, x + window_var)]))) < window_var |
                                                                any(sapply(get(day_var), function(x) !all(abs(diff(get(day_var)[between(get(day_var), x, x + window_var - 1)])) == 1))), NA_real_, 
                                                              first(sapply(get(day_var), function(x) round(sd(get(count_var)[between(get(day_var), x, x + window_var - 1)]),3)))), by = mget(group_var)]

  return(df)

}

【讨论】:

  • @arg0naut- 感谢您提供这个出色的功能。但是,这有点超出了我的 R 编码技能,并且我无法以一种允许我在未来复制的方式跟踪代码。例如,你能告诉我,我将如何将其转换为从每个分组变量的基数开始的累积时间窗口(即 -1 到 -2,-1 到 -3),而不是使用累积时间窗口,我可以从变量的顶部(即 -6 到 -5、-6 到 -4、-6 到 -3)。非常感谢。
  • 我尝试在“x - window_var”部分用减号代替加号,但这不起作用!我真的很想理解它,这样我就可以改进我的 R 编码!我通常以我的问题中的格式编写 for 循环,但我意识到这不是更有效的做事方式。感谢您的意见!
  • @jjulip 我添加了一个如何切换它的示例;我是在平板电脑上完成的,所以无法检查它是否有效,但应该可以。
  • 谢谢您,但是如果缺少一天,则返回 Has 的函数部分不太正确。在所有连续天的简单情况下,这很有效。谢谢。
  • 这似乎与我较大的数据集中存在第一天和第二天缺失的情况有关,因此代码不会在开头自动添加NA,这是一个不同的问题, 解决方案会有不同的规则,所以没问题。非常感谢您的帮助!
【解决方案2】:

考虑使用by 构建一个包含滚动meansd 计算的数据框列表。然后使用do.call 绑定列表中的所有数据框元素。

df_list <- by(df, df[c("Animal", "transactionID")], function(sub)
  transform(sub,
            max_day = min(sub$Day),
            mean = sapply(sub$Day, function(i) mean(sub[sub$Day >= i,]$Count)),
            sd = sapply(sub$Day, function(i) sd(sub[sub$Day >= i,]$Count))
  )
)
# BIND ALL DF ELEMENTS INTO ONE (FILTERING OUT NULL ELEMENTS) 
newdf <- do.call(rbind, Filter(NROW, df_list))

newdf
#    Animal Count Day transactionID max_day      mean         sd
# 1       a    45  -6             1      -6  32.16667  19.343388
# 2       a    54  -5             1      -6  29.60000  20.452384
# 3       a    22  -4             1      -6  23.50000  17.597348
# 4       a     3  -3             1      -6  24.00000  21.517435
# 5       a    23  -2             1      -6  34.50000  16.263456
# 6       a    46  -1             1      -6  46.00000         NA
# 7       a    45  -5             2      -5  38.00000  18.828170
# 8       a    22  -4             2      -5  36.25000  21.266170
# 9       a    67  -3             2      -5  41.00000  23.302360
# 10      a    34  -2             2      -5  28.00000   8.485281
# 11      a    22  -1             2      -5  22.00000         NA
# 12      b    34  -4             3      -4 212.75000 310.240095
# 13      b   677  -3             3      -4 272.33333 350.816666
# 14      b    86  -2             3      -4  70.00000  22.627417
# 15      b    54  -1             3      -4  54.00000         NA
# 16      b     4  -3             4      -3  52.66667  47.088569
# 17      b    56  -2             4      -3  77.00000  29.698485
# 18      b    98  -1             4      -3  98.00000         NA
# 19      c    23  -6             5      -6  39.80000  24.833445
# 20      c    54  -5             5      -6  44.00000  26.545558
# 21      c    22  -3             5      -6  40.66667  31.469562
# 22      c    77  -2             5      -6  50.00000  38.183766
# 23      c    23  -1             5      -6  23.00000         NA

理想情况下,您将使用上述作为最终输出,因为它是长格式,因为许多数据分析方法都需要这种形状。但是,如果您确实需要宽格式,请在上面构建后添加reshapemerge

rdf <- reshape(newdf, idvar = c("Animal", "transactionID", "max_day"), drop = c("Count"),
               v.names = c("sd", "mean"), timevar = "Day", direction="wide")
# MERGE ORIGINAL DATA FRAME
rdf <- merge(df, rdf, by=c("Animal", "transactionID"))

# CLEAN UP AND REORDER COLUMNS
names(rdf) <- gsub(".-", "_d", names(rdf))
rdf <- rdf[,c(1:5, rev(6:ncol(rdf)))]

rdf        
#    Animal transactionID Count Day max_day mean_d1 sd_d1 mean_d2     sd_d2   mean_d3     sd_d3 mean_d4     sd_d4 mean_d5    sd_d5  mean_d6    sd_d6
# 1       a             1    45  -6      -6      46    NA    34.5 16.263456  24.00000  21.51743   23.50  17.59735    29.6 20.45238 32.16667 19.34339
# 2       a             1    54  -5      -6      46    NA    34.5 16.263456  24.00000  21.51743   23.50  17.59735    29.6 20.45238 32.16667 19.34339
# 3       a             1    22  -4      -6      46    NA    34.5 16.263456  24.00000  21.51743   23.50  17.59735    29.6 20.45238 32.16667 19.34339
# 4       a             1     3  -3      -6      46    NA    34.5 16.263456  24.00000  21.51743   23.50  17.59735    29.6 20.45238 32.16667 19.34339
# 5       a             1    23  -2      -6      46    NA    34.5 16.263456  24.00000  21.51743   23.50  17.59735    29.6 20.45238 32.16667 19.34339
# 6       a             1    46  -1      -6      46    NA    34.5 16.263456  24.00000  21.51743   23.50  17.59735    29.6 20.45238 32.16667 19.34339
# 7       a             2    45  -5      -5      22    NA    28.0  8.485281  41.00000  23.30236   36.25  21.26617    38.0 18.82817       NA       NA
# 8       a             2    22  -4      -5      22    NA    28.0  8.485281  41.00000  23.30236   36.25  21.26617    38.0 18.82817       NA       NA
# 9       a             2    67  -3      -5      22    NA    28.0  8.485281  41.00000  23.30236   36.25  21.26617    38.0 18.82817       NA       NA
# 10      a             2    34  -2      -5      22    NA    28.0  8.485281  41.00000  23.30236   36.25  21.26617    38.0 18.82817       NA       NA
# 11      a             2    22  -1      -5      22    NA    28.0  8.485281  41.00000  23.30236   36.25  21.26617    38.0 18.82817       NA       NA
# 12      b             3    34  -4      -4      54    NA    70.0 22.627417 272.33333 350.81667  212.75 310.24010      NA       NA       NA       NA
# 13      b             3   677  -3      -4      54    NA    70.0 22.627417 272.33333 350.81667  212.75 310.24010      NA       NA       NA       NA
# 14      b             3    86  -2      -4      54    NA    70.0 22.627417 272.33333 350.81667  212.75 310.24010      NA       NA       NA       NA
# 15      b             3    54  -1      -4      54    NA    70.0 22.627417 272.33333 350.81667  212.75 310.24010      NA       NA       NA       NA
# 16      b             4     4  -3      -3      98    NA    77.0 29.698485  52.66667  47.08857      NA        NA      NA       NA       NA       NA
# 17      b             4    56  -2      -3      98    NA    77.0 29.698485  52.66667  47.08857      NA        NA      NA       NA       NA       NA
# 18      b             4    98  -1      -3      98    NA    77.0 29.698485  52.66667  47.08857      NA        NA      NA       NA       NA       NA
# 19      c             5    23  -6      -6      23    NA    50.0 38.183766  40.66667  31.46956      NA        NA    44.0 26.54556 39.80000 24.83345
# 20      c             5    54  -5      -6      23    NA    50.0 38.183766  40.66667  31.46956      NA        NA    44.0 26.54556 39.80000 24.83345
# 21      c             5    22  -3      -6      23    NA    50.0 38.183766  40.66667  31.46956      NA        NA    44.0 26.54556 39.80000 24.83345
# 22      c             5    77  -2      -6      23    NA    50.0 38.183766  40.66667  31.46956      NA        NA    44.0 26.54556 39.80000 24.83345
# 23      c             5    23  -1      -6      23    NA    50.0 38.183766  40.66667  31.46956      NA        NA    44.0 26.54556 39.80000 24.83345

【讨论】:

  • 优秀。非常感谢。我接受了 arg0naut 的回答,因为它们确实包含了一种在缺少一天的情况后插入 NA 的方法,但我发现这个解决方案更容易解释为相对有限的 R 用户!谢谢。
猜你喜欢
  • 1970-01-01
  • 1970-01-01
  • 1970-01-01
  • 2011-02-06
  • 2010-12-12
  • 2014-12-21
  • 2016-09-05
  • 2015-07-02
  • 1970-01-01
相关资源
最近更新 更多