R：仅导入和合并特定列

Question

我有一个来自网络的 CSV URL 列表，并将它们合并到一个向量中。 现在，我想用read_csv 阅读这份清单。

例子：

files <- c("csv_link1.csv",
             "csv_link2.csv",
             "csv_link3.csv",
              and so on....) 
data <- map_dfr(files, read_csv)

这没问题。问题是在 CSV 文件中有填充了不同值的列。因此，例如，在 CSV1 中有列“V1”，用 double 填充，而在 CSV 中，同一列是“V1”，用字符填充。无法合并 CSV，因为它们是不同的数据类型。

就我而言，我认为解决这个问题有两种可能性。

1. 我只导入某些列，所以我说read_csv 只读取列（V2 和 V3）而不是 V1

或者

1. 我将列合并到与col_types 相同的数据类型

我都试过了，但由于语法正确而失败了。

我尝试了类似的东西

data <- map_dfr(files, read_csv(cols_only(the col names)))

但是，这行不通。

如何仅导入和合并特定列？

以我为例：

library(data.table)
library(readr)
library(purrr)


files <- c("https://www.football-data.co.uk/mmz4281/1920/EC.csv",
           "https://www.football-data.co.uk/mmz4281/1819/EC.csv",
           "https://www.football-data.co.uk/mmz4281/1718/EC.csv",
           "https://www.football-data.co.uk/mmz4281/1617/EC.csv",
           "https://www.football-data.co.uk/mmz4281/1516/EC.csv",
           "https://www.football-data.co.uk/mmz4281/1415/EC.csv",
           "https://www.football-data.co.uk/mmz4281/1314/EC.csv",
           "https://www.football-data.co.uk/mmz4281/1213/EC.csv",
           "https://www.football-data.co.uk/mmz4281/1112/EC.csv",
           "https://www.football-data.co.uk/mmz4281/1011/EC.csv")


data <- map_dfr(files, read_csv)

Error: Can't combine `BbAH` <character> and `BbAH` <double>.

所以我的列BbAH 有不同的数据类型。但我不需要这个专栏。 如果由于这种不同的数据类型问题，我可以选择在合并运行出错之前将要合并的列，那就太酷了。

Answer 1

由于您只需要这七个变量，因此您可以使用fread 读取这些特定变量以避免BbAH 变量出现问题。

library(data.table)
library(dplyr)
library(purrr)

files <- c("https://www.football-data.co.uk/mmz4281/1920/EC.csv",
           "https://www.football-data.co.uk/mmz4281/1819/EC.csv",
           "https://www.football-data.co.uk/mmz4281/1718/EC.csv",
           "https://www.football-data.co.uk/mmz4281/1617/EC.csv",
           "https://www.football-data.co.uk/mmz4281/1516/EC.csv",
           "https://www.football-data.co.uk/mmz4281/1415/EC.csv",
           "https://www.football-data.co.uk/mmz4281/1314/EC.csv",
           "https://www.football-data.co.uk/mmz4281/1213/EC.csv",
           "https://www.football-data.co.uk/mmz4281/1112/EC.csv",
           "https://www.football-data.co.uk/mmz4281/1011/EC.csv")

# Identify columns you need
myColumns = c("Date","Time","HomeTeam","AwayTeam","FTHG","FTAG","FTR")

# Modified function found in https://stackoverflow.com/a/51348578/8535855
# takes a filename and a vector of columns as input
fread_allfiles <- function(file, columns){
  x <- fread(file, select = columns) %>% 
    select(everything())   # 
  return(x)
}

df_all <- files %>% 
  map_df(~ fread_allfiles(.,myColumns))

head(df_all)

产生以下格式：

         Date  Time     HomeTeam       AwayTeam FTHG FTAG FTR
1: 03/08/2019 12:30    Stockport     Maidenhead    0    1   A
2: 03/08/2019 15:00    Aldershot          Fylde    1    2   A
3: 03/08/2019 15:00       Barnet         Yeovil    1    0   H
4: 03/08/2019 15:00 Chesterfield Dover Athletic    1    2   A
5: 03/08/2019 15:00      Chorley        Bromley    0    0   D
6: 03/08/2019 15:00  Dag and Red         Woking    0    2   A

然后，如果需要，您可以重新格式化 Date 和 Time 列。看起来第一个文件有 Time 的任何值？所以剩下的都填写为NA

> str(df_all)
Classes ‘data.table’ and 'data.frame':  5429 obs. of  7 variables:
 $ Date    : chr  "03/08/2019" "03/08/2019" "03/08/2019" "03/08/2019" ...
 $ Time    : chr  "12:30" "15:00" "15:00" "15:00" ...
 $ HomeTeam: chr  "Stockport" "Aldershot" "Barnet" "Chesterfield" ...
 $ AwayTeam: chr  "Maidenhead" "Fylde" "Yeovil" "Dover Athletic" ...
 $ FTHG    : int  0 1 1 1 0 0 1 1 2 1 ...
 $ FTAG    : int  1 2 0 2 0 2 0 4 2 3 ...
 $ FTR     : chr  "A" "A" "H" "A" ...
 - attr(*, ".internal.selfref")=<externalptr>

Answer 2

我们可以 select 阅读 csv 后您需要的列，并使用 map_df 组合它们。

library(tidyverse)
result <- map_df(files, ~read_csv(.x) %>% select(Date,HomeTeam,AwayTeam,FTHG,FTAG,FTR))

Answer 3

这个怎么样：

library(data.table)
library(readr)

rbindlist(lapply(files, read_csv, col_types = "character"))

这会将所有列导入为character，因此您需要在合并后将它们转换为您最初想要的任何内容。