R中包含子字符串的属性的Web抓取答案

【问题标题】：Web scraping in R of an attribute containing a substringR中包含子字符串的属性的Web抓取
【发布时间】：2016-08-09 05:51:05
【问题描述】：

我正在使用 R 中的 xml2 包从网页中抓取数据。我要抓取的文本由如下所示的标签括起来：

<td>
<a href="javascript:WebForm_DoPostBackWithOptions(new 
WebForm_PostBackOptions(&quot;ctl00$CenterContent$ctl01&quot;,
&quot;&quot;, true, &quot;&quot;, &quot;&quot;, false,
true))">Species A    
</a></td>
<td>
<a href="javascript:WebForm_DoPostBackWithOptions(new
WebForm_PostBackOptions(&quot;ctl00$CenterContent$ctl02&quot;,
&quot;&quot;, true, &quot;&quot;, &quot;&quot;, false,
true))">Species B   </a></td>
<td><a href="javascript:WebForm_DoPostBackWithOptions(new
WebForm_PostBackOptions(&quot;ctl00$CenterContent$ctl03&quot;,
&quot;&quot;, true, &quot;&quot;, &quot;&quot;, false,
true))">Sepcies C    </a></td>
<td>
<a href="javascript:WebForm_DoPostBackWithOptions(new
WebForm_PostBackOptions(&quot;ctl00$CenterContent$ctl04&quot;,
&quot;&quot;, true, &quot;&quot;, &quot;&quot;, false,
true))">Species D</a></td>
<td>
<a href="javascript:WebForm_DoPostBackWithOptions(new
WebForm_PostBackOptions(&quot;ctl00$CenterContent$ctl05&quot;,
&quot;&quot;, true, &quot;&quot;, &quot;&quot;, false,
true))">Species E    </a></td>

我尝试在 R 中使用以下代码行：

library(xml2)
page = read_html(website)
nodes = html_nodes(page, xpath='//td/a[@href*="javascript"]')

使用上面的代码，我只想提取具有包含子字符串“javascript”的 href 属性的所有节点，但我收到以下错误消息：

xmlXPathEval: evaluation failed
Warning message:
In xpath_search(x$node, x$doc, xpath = xpath, nsMap = ns, num_results = Inf) :
Invalid expression [1207]

如果有人有任何建议，我将不胜感激。

感谢大家的时间。

干杯。

【问题讨论】：

您是否使用xpath？否则，您可以这样做：jscripts <- page %>% rvest::html_nodes('td') %>% rvest::html_nodes('a') %>% rvest::html_attr('href') %>% .[grepl('javascript', .)]。您需要为管道运算符使用magrittr 包。

标签： r xpath rvest xml2

【解决方案1】：

您可以在xpath 中使用contains 来查找具有href 包含您感兴趣的文本的锚标记：

library(xml2)
library(rvest)
website <- '<td>
<a href="javascript:WebForm_DoPostBackWithOptions(new 
WebForm_PostBackOptions(&quot;ctl00$CenterContent$ctl01&quot;,
&quot;&quot;, true, &quot;&quot;, &quot;&quot;, false,
true))">Species A    
</a></td>
<td>
<a href="javascript:WebForm_DoPostBackWithOptions(new
WebForm_PostBackOptions(&quot;ctl00$CenterContent$ctl02&quot;,
&quot;&quot;, true, &quot;&quot;, &quot;&quot;, false,
true))">Species B   </a></td>
<td><a href="javascript:WebForm_DoPostBackWithOptions(new
WebForm_PostBackOptions(&quot;ctl00$CenterContent$ctl03&quot;,
&quot;&quot;, true, &quot;&quot;, &quot;&quot;, false,
true))">Sepcies C    </a></td>
<td>
<a href="javascript:WebForm_DoPostBackWithOptions(new
WebForm_PostBackOptions(&quot;ctl00$CenterContent$ctl04&quot;,
&quot;&quot;, true, &quot;&quot;, &quot;&quot;, false,
true))">Species D</a></td>
<td>
<a href="javascript:WebForm_DoPostBackWithOptions(new
WebForm_PostBackOptions(&quot;ctl00$CenterContent$ctl05&quot;,
&quot;&quot;, true, &quot;&quot;, &quot;&quot;, false,
true))">Species E    </a></td>'
page <- read_html(website)
nodes <- html_nodes(page, xpath='//td/a[contains(@href,"javascript")]')

> nodes
{xml_nodeset (5)}
[1] <a href="javascript:WebForm_DoPostBackWithOptions(new &#10;WebForm_PostBackOptions(&quot;ctl00$CenterConte ...
[2] <a href="javascript:WebForm_DoPostBackWithOptions(new&#10;WebForm_PostBackOptions(&quot;ctl00$CenterConten ...
[3] <a href="javascript:WebForm_DoPostBackWithOptions(new&#10;WebForm_PostBackOptions(&quot;ctl00$CenterConten ...
[4] <a href="javascript:WebForm_DoPostBackWithOptions(new&#10;WebForm_PostBackOptions(&quot;ctl00$CenterConten ...
[5] <a href="javascript:WebForm_DoPostBackWithOptions(new&#10;WebForm_PostBackOptions(&quot;ctl00$CenterConten ...
>

【讨论】：