R Web Scraping - 数据框答案

【问题标题】：R Web Scraping - data frameR Web Scraping - 数据框
【发布时间】：2018-04-16 22:52:43
【问题描述】：

我正在尝试使用以下变量创建数据框。但是，在使用 SelectorGadget 工具确定抓取此信息所需的 CSS 选择器后，向量会产生不同的值。即使直接从 HTML 源代码中复制了选择器。如果操作正确，该表应该有 34 行。这是我的代码和相应的错误：

womens_bb <- read_html("http://gomason.com/schedule.aspx?path=wbball")

womens_opponents <- womens_bb %>%
html_nodes(".sidearm-schedule-game-opponent-name a") %>%
html_text()

womens_locations <- womens_bb %>%
html_nodes(".sidearm-schedule-game-location span:nth-child(1)") %>%
html_text()

womens_dates <- womens_bb %>%
html_nodes(".sidearm-schedule-game-opponent-date span:nth-child(1)") %>%
html_text() 

womens_times <- womens_bb %>%
html_nodes(".sidearm-schedule-game-opponent-date span:nth-child(2)") %>%
html_text()
as.numeric()

womens_scores <- womens_bb %>%
html_nodes("div.sidearm-schedule-game-result span:nth-child(3)") %>%
html_text()
as.numeric() 

womens_win_loss <- womens_bb %>%
html_nodes(".text-italic span:nth-child(2)") %>%
html_text() %>%
str_replace("\\,", "")

womens_df <- data_frame(
  date = womens_dates, time = womens_times, opponent = womens_opponents, location = womens_locations, score = womens_scores, win_loss = womens_win_loss)

Error: Columns `date`, `time`, `opponent`, `score`, `win_loss` must be length 1 or 37, not 36, 36, 34, 34, 35

我该如何解决这个问题？

【问题讨论】：

标签： r web-scraping rvest

【解决方案1】：

我认为 img 标签存在一些问题。所以为了避免这些，你可以首先收集全局 div 标签（当我执行脚本时为 36 ），然后在里面做一个循环来得到你的结果。如果控制出现怪异的标签，请执行一些操作：

womens_bb <- read_html("http://gomason.com/schedule.aspx?path=wbball")
divs <- womens_bb %>% html_nodes(".sidearm-schedule-game")

for (div in divs){

  womens_opponents <- div %>%
    html_nodes(".sidearm-schedule-game-opponent-name, .sidearm-schedule-game-opponent-name a") %>%
    html_text
  womens_opponents <- gsub("\\s{2,}","",womens_opponents[1])

  womens_locations <- div %>%
    html_nodes(".sidearm-schedule-game-location span:nth-child(1)") %>%
    html_text()
  womens_locations <- womens_locations[1]

  womens_dates <- div %>%
    html_nodes(".sidearm-schedule-game-opponent-date span:nth-child(1)") %>%
    html_text() 

  womens_times <- div %>%
    html_nodes(".sidearm-schedule-game-opponent-date span:nth-child(2)") %>%
    html_text()

  womens_scores <- div %>%
    html_nodes("div.sidearm-schedule-game-result span:nth-child(3)") %>%
    html_text()
  if(length(womens_scores)==0) womens_scores = ""

  womens_win_loss <- div %>%
    html_nodes(".text-italic span:nth-child(2)") %>%
    html_text()
  womens_win_loss <-   gsub("\\,", "",womens_win_loss)  

  res <- c(date = womens_dates, time = womens_times, opponent = womens_opponents, location = womens_locations, score = womens_scores, win_loss = womens_win_loss)    
  print(length(res))
  df <- rbind(df,res)
}

希望对你有帮助，

哥达维亚诺尼

【讨论】：