使用 purrr:map() 从不规则列表中提取数据答案

【问题标题】：Extracting data from irregular lists using purrr:map()使用 purrr:map() 从不规则列表中提取数据
【发布时间】：2019-12-22 05:47:16
【问题描述】：

给定一个包含多个元素的列表，目标是将它们放入数据框中。 purr 包中的 map_df 函数对于常规列表非常有用，但对于不规则列表会出错。

例如，按照this 教程，以下工作：

library(purrr)
library(repurrrsive) # The data comes from this package


map_dfr(got_chars, magrittr::extract, c("name", "culture", "gender", "id", "born", "alive"))

 A tibble: 30 x 6
   name               culture  gender    id born                                   alive
   <chr>              <chr>    <chr>  <int> <chr>                                  <lgl>
 1 Theon Greyjoy      Ironborn Male    1022 In 278 AC or 279 AC, at Pyke           TRUE 
 2 Tyrion Lannister   ""       Male    1052 In 273 AC, at Casterly Rock            TRUE 
 3 Victarion Greyjoy  Ironborn Male    1074 In 268 AC or before, at Pyke           TRUE 
 4 Will               ""       Male    1109 ""                                     FALSE
 5 Areo Hotah         Norvoshi Male    1166 In 257 AC or before, at Norvos         TRUE 
 6 Chett              ""       Male    1267 At Hag's Mire                          FALSE
 7 Cressen            ""       Male    1295 In 219 AC or 220 AC                    FALSE
 8 Arianne Martell    Dornish  Female   130 In 276 AC, at Sunspear                 TRUE 
 9 Daenerys Targaryen Valyrian Female  1303 In 284 AC, at Dragonstone              TRUE 
10 Davos Seaworth     Westeros Male    1319 In 260 AC or before, at King's Landing TRUE 
# … with 20 more rows

但是，如果从列表中删除一个元素，则该函数将失败。

got_chars[[1]]["gender"]<-NULL
map_dfr(got_chars, magrittr::extract, c("name", "culture", "gender", "id", "born", "alive"))

#Error: Argument 3 is a list, must contain atomic vectors

所需的输出将是缺少元素的NA 值。什么是优雅的解决方案？我怀疑解决方案包括使用purrr:possibly()，但我还没有弄清楚。

【问题讨论】：

您可以尝试添加一个步骤，从结果with Filter() as in this answer 中删除NULL 值，因为bind_rows() 将填入NA。我可能会为此创建一个新功能，但使用波浪号编码可能看起来像map_dfr(got_chars, ~magrittr::extract(.x, c("name", "culture", "gender", "id", "born", "alive") ) %>% Filter(Negate(is.null), .) )

标签： r list dictionary purrr

【解决方案1】：

tidyr 的开发版本具有强大的新“取消嵌套”功能，它们可以处理这些有问题的数据（选项 1）。解决此问题的另一种方法是逐列解决问题，它允许您使用 .default 参数到 purrr::map()，它提供了一个用于缺失元素的值（选项 2）。

library(tidyverse)   # purrr, tidyr, and dplyr
library(repurrrsive) # The data comes from this package

got_chars_mutilated <- got_chars
got_chars_mutilated[[1]]["gender"] <- NULL

# original problem
map_dfr(
  got_chars_mutilated,
  magrittr::extract,
  c("name", "culture", "gender", "id", "born", "alive")
)
#> Error: Argument 3 is a list, must contain atomic vectors

# Option 1:
# expanded unnest_*() functions coming soon in tidyr
packageVersion("tidyr")
#> [1] '0.8.99.9000'

# automatic unnesting leads to ... unnest_wider()
tibble(got = got_chars_mutilated) %>% 
  unnest_auto(got)
#> Using `unnest_wider(got)`; elements have {n_common} names in common
#> # A tibble: 30 x 18
#>    url      id name  culture born  died  alive titles aliases father mother
#>    <chr> <int> <chr> <chr>   <chr> <chr> <lgl> <list> <list>  <chr>  <chr> 
#>  1 http…  1022 Theo… Ironbo… In 2… ""    TRUE  <chr … <chr [… ""     ""    
#>  2 http…  1052 Tyri… ""      In 2… ""    TRUE  <chr … <chr [… ""     ""    
#>  3 http…  1074 Vict… Ironbo… In 2… ""    TRUE  <chr … <chr [… ""     ""    
#>  4 http…  1109 Will  ""      ""    In 2… FALSE <chr … <chr [… ""     ""    
#>  5 http…  1166 Areo… Norvos… In 2… ""    TRUE  <chr … <chr [… ""     ""    
#>  6 http…  1267 Chett ""      At H… In 2… FALSE <chr … <chr [… ""     ""    
#>  7 http…  1295 Cres… ""      In 2… In 2… FALSE <chr … <chr [… ""     ""    
#>  8 http…   130 Aria… Dornish In 2… ""    TRUE  <chr … <chr [… ""     ""    
#>  9 http…  1303 Daen… Valyri… In 2… ""    TRUE  <chr … <chr [… ""     ""    
#> 10 http…  1319 Davo… Wester… In 2… ""    TRUE  <chr … <chr [… ""     ""    
#> # … with 20 more rows, and 7 more variables: spouse <chr>,
#> #   allegiances <list>, books <list>, povBooks <list>, tvSeries <list>,
#> #   playedBy <list>, gender <chr>

# let's do it again, calling the proper function, and inspect `gender`
tibble(got = got_chars_mutilated) %>% 
  unnest_wider(got) %>% 
  pull(gender)
#>  [1] NA       "Male"   "Male"   "Male"   "Male"   "Male"   "Male"  
#>  [8] "Female" "Female" "Male"   "Female" "Male"   "Female" "Male"  
#> [15] "Male"   "Male"   "Female" "Female" "Female" "Male"   "Male"  
#> [22] "Male"   "Male"   "Male"   "Male"   "Female" "Male"   "Male"  
#> [29] "Male"   "Female"

# Option 2:
# attack this column-wise
# mapping the names gives access to the `.default` argument for missing elements
c("name", "culture", "gender", "id", "born", "alive") %>% 
  set_names() %>% 
  map(~ map(got_chars_mutilated, .x, .default = NA)) %>%
  map(simplify) %>% 
  as_tibble()
#> # A tibble: 30 x 6
#>    name           culture  gender      id born                        alive
#>    <chr>          <chr>    <list>   <int> <chr>                       <lgl>
#>  1 Theon Greyjoy  Ironborn <lgl [1…  1022 In 278 AC or 279 AC, at Py… TRUE 
#>  2 Tyrion Lannis… ""       <chr [1…  1052 In 273 AC, at Casterly Rock TRUE 
#>  3 Victarion Gre… Ironborn <chr [1…  1074 In 268 AC or before, at Py… TRUE 
#>  4 Will           ""       <chr [1…  1109 ""                          FALSE
#>  5 Areo Hotah     Norvoshi <chr [1…  1166 In 257 AC or before, at No… TRUE 
#>  6 Chett          ""       <chr [1…  1267 At Hag's Mire               FALSE
#>  7 Cressen        ""       <chr [1…  1295 In 219 AC or 220 AC         FALSE
#>  8 Arianne Marte… Dornish  <chr [1…   130 In 276 AC, at Sunspear      TRUE 
#>  9 Daenerys Targ… Valyrian <chr [1…  1303 In 284 AC, at Dragonstone   TRUE 
#> 10 Davos Seaworth Westeros <chr [1…  1319 In 260 AC or before, at Ki… TRUE 
#> # … with 20 more rows

^{由reprex package (v0.3.0.9000) 于 2019 年 8 月 15 日创建}

【讨论】：

【解决方案2】：

一种方法是定义一个partial()ly-specified pluck() 来提取感兴趣的名称，如果它丢失则返回NA。将修改后的 pluck() 传递给双映射，内部映射遍历要提取的名称，外部映射遍历您的 got_chars 列表：

v <- set_names(c("name", "culture", "gender", "id", "born", "alive"))
map_dfr( got_chars, ~map(v, partial(pluck, .x, .default=NA)) )
# # A tibble: 30 x 6
#    name             culture  gender    id born                             alive
#    <chr>            <chr>    <chr>  <int> <chr>                            <lgl>
#  1 Theon Greyjoy    Ironborn NA      1022 In 278 AC or 279 AC, at Pyke     TRUE 
#  2 Tyrion Lannister ""       Male    1052 In 273 AC, at Casterly Rock      TRUE 
#  3 Victarion Greyj… Ironborn Male    1074 In 268 AC or before, at Pyke     TRUE 
#  4 Will             ""       Male    1109 ""                               FALSE
#  5 Areo Hotah       Norvoshi Male    1166 In 257 AC or before, at Norvos   TRUE 
#  6 Chett            ""       Male    1267 At Hag's Mire                    FALSE
#  7 Cressen          ""       Male    1295 In 219 AC or 220 AC              FALSE
#  8 Arianne Martell  Dornish  Female   130 In 276 AC, at Sunspear           TRUE 
#  9 Daenerys Targar… Valyrian Female  1303 In 284 AC, at Dragonstone        TRUE 
# 10 Davos Seaworth   Westeros Male    1319 In 260 AC or before, at King's … TRUE 
# # … with 20 more rows

澄清一下，.x 迭代了got_chars，因为它存在于~ 指定的lambda 函数中，因此它对应于外部map。内部map 的函数由partial() 指定，它将当前查看的got_chars 元素（即.x）作为第一个参数附加到pluck()。修改后的pluck() 然后接受要提取的名称作为其（新）第一个参数，因此它可以按原样传递给内部映射，而无需任何额外的~。

【讨论】：

【解决方案3】：

一个固有的问题是[（或其别名magrittr::extract）在我们尝试提取的元素不存在时的行为：

list(a = 1)["b"]
# $<NA>
# NULL

magrittr::extract(list(a = 1), "b")
# $<NA>
# NULL

我们可以定义：

extract_if_present <- function(x, y) {
  x[intersect(y, names(x))]
}

表现如下：

extract_if_present(list(a = 1), "b")
# named list()

然后对缺少元素的行绑定“正常工作”：

map_dfr(
  got_chars_mutilated,
  extract_if_present,
  c("name", "culture", "gender", "id", "born", "alive")
)
# # A tibble: 30 x 6
#    name               culture     id born                                   alive gender
#    <chr>              <chr>    <int> <chr>                                  <lgl> <chr> 
#  1 Theon Greyjoy      Ironborn  1022 In 278 AC or 279 AC, at Pyke           TRUE  NA    
#  2 Tyrion Lannister   ""        1052 In 273 AC, at Casterly Rock            TRUE  Male  
#  3 Victarion Greyjoy  Ironborn  1074 In 268 AC or before, at Pyke           TRUE  Male  
#  4 Will               ""        1109 ""                                     FALSE Male  
#  5 Areo Hotah         Norvoshi  1166 In 257 AC or before, at Norvos         TRUE  Male  
#  6 Chett              ""        1267 At Hag's Mire                          FALSE Male  
#  7 Cressen            ""        1295 In 219 AC or 220 AC                    FALSE Male  
#  8 Arianne Martell    Dornish    130 In 276 AC, at Sunspear                 TRUE  Female
#  9 Daenerys Targaryen Valyrian  1303 In 284 AC, at Dragonstone              TRUE  Female
# 10 Davos Seaworth     Westeros  1319 In 260 AC or before, at King's Landing TRUE  Male  
# # … with 20 more rows

列的顺序有点混乱，取决于行的顺序和它们遗漏的内容。

【讨论】：

【解决方案4】：

喜欢那个教程！在教程的最后作者说：

在编程时，以通常的方式显式指定类型并构建数据框更安全，但也更麻烦。

您可以使用更详细的方式将默认设置为 NA

got_chars %>% {
  tibble(
    name = map_chr(., "name"),
    culture = map_chr(., "culture"),
    gender = map_chr(., "gender", .default = NA),
    id = map_chr(., "id"),
    born = map_chr(., "born"),
    alive = map_chr(., "alive")
  )
}
# # A tibble: 30 x 6
# name               culture    gender id    born                                     alive
# <chr>              <chr>      <chr>  <chr> <chr>                                    <chr>
#   1 Theon Greyjoy      "Ironborn" NA     1022  "In 278 AC or 279 AC, at Pyke"           TRUE 
# 2 Tyrion Lannister   ""         Male   1052  "In 273 AC, at Casterly Rock"            TRUE 
# 3 Victarion Greyjoy  "Ironborn" Male   1074  "In 268 AC or before, at Pyke"           TRUE 
# 4 Will               ""         Male   1109  ""                                       FALSE
# 5 Areo Hotah         "Norvoshi" Male   1166  "In 257 AC or before, at Norvos"         TRUE 
# 6 Chett              ""         Male   1267  "At Hag's Mire"                          FALSE
# 7 Cressen            ""         Male   1295  "In 219 AC or 220 AC"                    FALSE
# 8 Arianne Martell    "Dornish"  Female 130   "In 276 AC, at Sunspear"                 TRUE 
# 9 Daenerys Targaryen "Valyrian" Female 1303  "In 284 AC, at Dragonstone"              TRUE 
# 10 Davos Seaworth     "Westeros" Male   1319  "In 260 AC or before, at King's Landing" TRUE

【讨论】：