【问题标题】:using rvest to create a data.frame使用 rvest 创建一个 data.frame
【发布时间】:2021-06-07 10:58:02
【问题描述】:

我正在学习如何使用 rvest 抓取网站,运行以下代码后得到以下输出:

p<-read_html("https://example.com") %>%
html_text()
p

[1] "window.location = "https://example.com" 提交信息: 名称:\n\t\t\t\t\t Jon Doe 市:\n\t\t\t\t\t 密尔沃基 州:\n\t\t\t\t\t WI 邮编:\n\t\t\t\t\t 53045 汽车信息:日期 购买时间:\n\t\t\t\t\t 07/13/1999 时间:\n\t\t\t\t\t 8 pm 品牌:\n\t\t\t\t\t 福特 颜色:\n\t\t\t\t\t 蓝色

我想创建一个数据集(data.frame?),其中包含以下信息(这只是第一行):

Name City State Zip Date Bought Time Brand Color
Jon Doe Milwaukee WI 53045 09/13/1999 8 pm Ford Blue

请注意,我从上面删除了 "window.location = \"https://example.com"Submission Information:Car Information:\n\t\t\t\t\t 之类的内容。使用 rvest 抓取此文本输出后,我将如何在 R 中执行此操作?

这不是一个公共网站,所以我不能分享实际的链接——它需要经过身份验证的访问,我已经弄清楚了。但我能够抓取我试图抓取的基本页面的样本。我基本上想要一个包含以下所有信息的表格,减去会员信息、汽车信息、经销商信息、买家信息和保险信息,因为这些标签只是标识数据集不需要的部分。

<tr><td><span style="color:#ffb76b;"> Member Information:</span></td><br />
<td><span style="margin-left:145px; color:white;"> Name:</span></td>
<td><span style="color:white; padding-left:100px;"> John Doe</span></td><br /><td><span style="margin-left:145px; color:white;"> City:</span></td>
<td><span style="color:white; padding-left:112px;"> Milwaukee</span></td><br /><td><span style="margin-left:145px; color:white;"> State:</span></td>
<td><span style="color:white; padding-left:107px;"> WI</span></td><br /><td><span style="margin-left:145px; color:white;"> Zip:</span></td>
<td><span style="color:white; padding-left:118px;"> 53045</span></td><br /><td><span style="margin-left:145px; color:white;"> Angler Class:</span></td>
<td><span style="color:white; padding-left:62px;"> Male</span></td><br /><td><span style="color:#ffb76b"> Car Information:</span></td><br />
<td><span style="margin-left:145px; color:white;"> Date Bought:</span></td>
<td><span style="color:#ffb76b; padding-left:62px;"> 09/13/1999</span></td><br /><td><span style="margin-left:145px; color:white;"> Time:</span></td>
<td><span style="color:white; padding-left:107px;"> 8 pm</span></td><br /><td><span style="margin-left:145px; color:white;"> Length:</span></td>
<td><span style="color:white; padding-left:97px;"> 16ft</span></td><br /><td><span style="margin-left:145px; color:white;"> Weight:</span></td>
<td><span style="color:white; padding-left:95px;"> Not Specified</span></td><br /><td><span style="margin-left:145px; color:white;"> Tire:</span></td>
<td><span style="color:white; padding-left:108px;"> Not Specified</span></td><br /><td><span style="margin-left:145px; color:white;"> Age:</span></td>
<td><span style="color:white; padding-left:72px;"> 20 years</span></td><br /><td><span style="margin-left:145px; color:white;"> Mileage:</span></td>
<td><span style="color:white; padding-left:67px;"> 65,305</span></td><br /><td><span style="margin-left:145px; color:white;"> Damage:</span></td>
<td><span style="color:white; padding-left:58px;"> None</span></td><br /><td><span style="margin-left:145px; color:white;"> Model:</span></td>
<td><span style="color:white; padding-left:93px;"> Taurus</span></td><br /><td><span style="color:#ffb76b"> Dealer Information:</span></td><br />
<td><span style="margin-left:145px; color:white;"> Name:</span></td>
<td><span style="color:white; padding-left:114px;"> ABC Auto</span></td><br /><td><span style="margin-left:145px; color:white;"> Zip:</span></td>
<td><span style="color:white; padding-left:47px;"> 15101</span></td><br /><td><span style="margin-left:145px; color:white;"> Rating:</span></td>
<td><span style="color:white; padding-left:63px;"> 5</span></td><br /><td><span style="color:#ffb76b"> Buyer Information:</span></td><br />
<td><span style="margin-left:145px; color:white;"> Sales Person:</span></td>
<td><span style="color:white; padding-left:66px;"> John Wick</span></td><br /><td><span style="margin-left:145px; color:white;"> County:</span></td>
<td><span style="color:white; padding-left:94px;"> Waukesha</span></td><br /><td><span style="margin-left:145px; color:white;"> State:</span></td>
<td><span style="color:white; padding-left:107px;"> Wisconsin</span></td><br /><td><span style="margin-left:145px; color:white;"> Rating:</span></td>
<td><span style="color:white; padding-left:58px;"> Good</span></td><br /><td><span style="margin-left:145px; color:white;"> Experience:</span></td>
<td><span style="color:white; padding-left:83px;"> Not Specified</span></td><br /><td><span style="color:#ffb76b"> Insurance Information:</span></td><br />
<td><span style="margin-left:145px; color:white;"> Insurer Name:</span></td>
<td><span style="color:white; padding-left:68px;"> Allstate</span></td><br /><td><span style="margin-left:145px; color:white;"> Primary Coverage:</span></td>
<td><span style="color:white; padding-left:52px;"> Tier3</span></td><br /><td><span style="margin-left:145px; color:white;"> Secondary Coverage:</span></td>
<td><span style="color:white; padding-left:34px;"> Not Specified</span></td><br /><td><span style="margin-left:145px; color:white;"> Policy Period:</span></td>
<td><span style="color:white; padding-left:64px;"> 2019</span></td><br /></tr></table>

【问题讨论】:

    标签: r web-scraping rvest


    【解决方案1】:
    data <-  read_html("https://example.com")
    
    html_nodes(data, "table") %>% 
      html_table()
    
    

    https://fahimahmad.netlify.app/posts/scraping-html-tables-into-r-data-frames/

    【讨论】:

    • 这样做只会在控制台中返回“list()”,而不是任何抓取的数据。也许我错过了什么?
    猜你喜欢
    • 2019-03-21
    • 2012-03-21
    相关资源
    最近更新 更多