识别具有两个固定值且在 r 范围内的重复项答案

【问题标题】：Identify duplicates with two fixed values and within a range in r识别具有两个固定值且在 r 范围内的重复项
【发布时间】：2022-01-20 04:07:40
【问题描述】：

我有大量的电子邮件数据框，如下所示：

structure(list(messagesubject = c("Re: Fuentes/referencias para tutorial YoPeatón", 
"Re: Fuentes/referencias para tutorial YoPeatón", "Protejamos nuestras playas", 
"Protejamos nuestras playas", "Protejamos nuestras playas", "Protejamos nuestras playas", 
"Protejamos nuestras playas", "Protejamos nuestras playas", "Protejamos nuestras playas", 
"Protejamos nuestras playas", "documento", "documento", "documento"
), senderaddress = c("6607", "7998", "4799", "9731", "4799", 
"79", "9731", "4799", "9731", "4799", "1846", "1846", "1846"
), recipient = c("94529", "163910", "289", "289", "270177", "2310", 
"1118", "1118", "2551", "2551", "753", "12291", "106610"), datetimesent = structure(c(1515632629, 
1515632629, 1515632636, 1515632636, 1515632639, 1515632640, 1515632641, 
1515632641, 1515632647, 1515632647, 1515632680, 1515632680, 1515632680
), tzone = "UTC", class = c("POSIXct", "POSIXt"))), row.names = c(NA, 
-13L), class = c("tbl_df", "tbl", "data.frame"))

我需要提取具有相同“邮件主题”、相同“发件人地址”且在 5 秒时间范围内（“日期时间发送”）的电子邮件。我正在清理一个数据集，很可能这些只是同一封电子邮件，但交付时有延迟。

我能够提取精确的重复：

d[(duplicated(d[c(4,2,1)]) | duplicated(d[c(4,2,1)], fromLast = TRUE)), ]

但考虑到“datetimesent”中的时间范围，我不知道该怎么做

结果应该有第 3、5、8 行（具有相同的发件人地址、相同的消息主题，并且它们之间的最长 5 秒，以及第 4 和第 7 行。它应该看起来像：

messagesubject             senderaddress recipient datetimesent       
              
1 Protejamos nuestras playas 4799          289       2018-01-11 01:03:56
2 Protejamos nuestras playas 9731          289       2018-01-11 01:03:56
3 Protejamos nuestras playas 4799          270177    2018-01-11 01:03:59
4 Protejamos nuestras playas 9731          1118      2018-01-11 01:04:01
5 Protejamos nuestras playas 4799          1118      2018-01-11 01:04:01

【问题讨论】：

一条消息只能复制一次（仅限 2x 条消息）吗？为什么不按收件人分组？
不，它可以出现多次。我在数据中看到了超过 200 个案例。我不按收件人分组，因为可以有不同的收件人。我可以同时向许多人发送一封电子邮件。我的猜测是电子邮件被卡在服务器中，因此它一次发送一封电子邮件，时差很小，而不是一起发送。这就是我想要解决的问题
为什么“documento”消息没有出现在您的预期结果中？他们应该符合标准吗？

标签： r time duplicates range

【解决方案1】：

我认为这是你想要的：

# first ensure file is in order
df<-df %>% arrange(senderaddress, datetimesent)

# if the sender address is the same, test if subject is the same
df$diffsub<-ifelse(df$senderaddress==data.table::shift(df$senderaddress,1), ifelse(df$messagesubject!=data.table::shift(df$messagesubject,1),"Different","Same Subj"),NA) 


# if diffSub is "Same Subj" compute time difference   
df$difftime<-ifelse(df$diffsub=="Same Subj",df$datetimesent-data.table::shift(df$datetimesent,1),NA)  

# if diffsub=Same Subj, email is also the same, so
df # examine

【讨论】：

【解决方案2】：

我们可以在 tidyverse 中使用分组来做到这一点。关键是按主题和地址分组，按时间排序，然后找出与上一条和下一条消息的时间差。

然后，我们可以过滤具有多行的组，并在其前后

library(dplyr)
library(lubridate)

d %>%
    group_by(messagesubject, senderaddress) %>%
    arrange(datetimesent) %>%
    mutate(lag = (datetimesent-lag(datetimesent)),     # time from previous msg
           lead = lead(datetimesent)-datetimesent) %>% # time to next msg
    filter(lag <= seconds(5) | lead <= seconds(5), # require msg within 5sec
           n() > 1)                                # require >1 msg with sub/add

# A tibble: 8 × 6
# Groups:   messagesubject, senderaddress [3]
  messagesubject             senderaddress recipient datetimesent        lag     lead   
  <chr>                      <chr>         <chr>     <dttm>              <drtn>  <drtn> 
1 Protejamos nuestras playas 4799          289       2018-01-11 01:03:56 NA secs  3 secs
2 Protejamos nuestras playas 9731          289       2018-01-11 01:03:56 NA secs  5 secs
3 Protejamos nuestras playas 4799          270177    2018-01-11 01:03:59  3 secs  2 secs
4 Protejamos nuestras playas 9731          1118      2018-01-11 01:04:01  5 secs  6 secs
5 Protejamos nuestras playas 4799          1118      2018-01-11 01:04:01  2 secs  6 secs
6 documento                  1846          753       2018-01-11 01:04:40 NA secs  0 secs
7 documento                  1846          12291     2018-01-11 01:04:40  0 secs  0 secs
8 documento                  1846          106610    2018-01-11 01:04:40  0 secs NA secs

【讨论】：