【问题标题】:Need an example of how to scrape this site需要一个如何抓取这个网站的例子
【发布时间】:2021-12-31 11:12:57
【问题描述】:

我的问题有点不清楚,但那是因为现阶段我真的不知道如何更清楚地问它.. 如果我得到答案,我会更准确地重命名它。 我是一个完全的新手,只是学习如何去做。 我正在尝试从this site 中仅抓取一个值

library("rvest")
url <- "https://www.fxblue.com/market-data/tools/sentiment"
web <- read_html(url)  
nodes <-  html_nodes(web,".SentimentValueCaptionLong")

得到

 html_text(nodes)
character(0)

我的下一次尝试

library(RSelenium)
rD <- rsDriver(browser="chrome",port=0999L,verbose = F,chromever = "95.0.4638.54")
remDr <- rD[["client"]]
remDr$maxWindowSize()
remDr$navigate("https://www.fxblue.com/market-data/tools/sentiment")
html <- remDr$getPageSource()[[1]]
page <- read_html(html)
nodes <- html_nodes(page, ".SentimentValueCaptionLong")

也一样

html_text(nodes)
character(0)

谁能告诉我如何正确地做,并解释你做了什么

【问题讨论】:

    标签: r web-scraping rvest rselenium


    【解决方案1】:
        library(rvest)
        library(dplyr)
        library(RSelenium)
        driver = rsDriver(browser = c("firefox"))
        remDr <- driver[["client"]]
        remDr$navigate(url)
    

    获取名称

    remDr$getPageSource()[[1]] %>% 
      read_html() %>% html_nodes('.SentimentRowCaption') %>% 
      html_text()
     [1] "AUD/CAD" "AUD/JPY" "AUD/NZD" "AUD/USD" "CAD/JPY" "DAX"     "EUR/AUD" "EUR/CAD" "EUR/CHF" "EUR/GBP" "EUR/JPY" "EUR/USD" "GBP/AUD" "GBP/CAD" "GBP/CHF"
    [16] "GBP/JPY" "GBP/USD" "NZD/USD" "USD/CAD" "USD/CHF" "USD/JPY" "XAU/USD"
    

    获取长数据

    remDr$getPageSource()[[1]] %>% 
      read_html() %>% html_nodes('.SentimentValueCaptionLong') %>% 
      html_text()
     [1] "79.2%" "38.4%" "56.1%" "68.9%" "26.8%" "28.7%" "68.7%" "79.5%" "80.7%" "85.3%" "57.0%" "76.4%" "36.1%" "67.4%" "69.7%" "54.9%" "82.3%" "65.1%" "25.0%"
    [20] "28.7%" "17.9%" "82.8%"
    

    获取简短数据

    remDr$getPageSource()[[1]] %>% 
      read_html() %>% html_nodes('.SentimentValueCaptionShort') %>% 
      html_text()
     [1] "20.8%" "61.4%" "43.5%" "31.3%" "73.8%" "70.8%" "31.7%" "20.0%" "19.9%" "14.3%" "43.5%" "23.4%" "64.0%" "32.2%" "30.0%" "45.8%" "17.7%" "34.8%" "74.5%"
    [20] "71.3%" "82.2%" "17.0%"
    

    【讨论】:

    • 非常感谢,它对我有用.. 你能解释一下我的问题是什么吗?
    • 您的代码有效。想知道为什么你没有得到结果。
    • 我也看不懂,大概是我做错了,请原谅我提出的愚蠢问题,我刚开始学习爬虫,对此我仍然很困惑......非常感谢为您的帮助!
    猜你喜欢
    • 1970-01-01
    • 1970-01-01
    • 2011-07-03
    • 2012-12-16
    • 1970-01-01
    • 1970-01-01
    • 2020-08-12
    • 1970-01-01
    相关资源
    最近更新 更多