正如@jihoward 所说，RSelenium 将解决这个问题，并且不需要检查底层网站的净流量/解剖来找到合适的数量。我还要注意，如果phantomjs 安装在用户系统上，RSelenium 可以在没有Selenium Server 的情况下运行。在这种情况下RSelenium 可以直接驱动phantomjs。 http://cran.r-project.org/web/packages/RSelenium/vignettes/RSelenium-headless.html有一个与无头浏览相关的小插曲
使用浏览器检查网络流量
在这种情况下，但是对流量的检查会产生以下名为 http://spiderbook.com/company/details/docs?rel=300795&docs_page=0 的 json 文件，它不受 cookie 保护或对用户代理字符串等不敏感。在这种情况下，可以执行以下操作：
library(RJSONIO)
res <- fromJSON("http://spiderbook.com/company/details/docs?rel=300795&docs_page=0")
> sapply(res$data, "[[", "url")
[1] "http://www.automotiveworld.com/news-releases/volkswagen-selects-bosch-chargepoint-e-golf-charging-solution-providers/"
[2] "http://insideevs.com/category/vw/"
[3] "http://www.greencarcongress.com/2014/07/20140711-vw.html"
[4] "http://www.vdubnews.com/volkswagen-chooses-bosch-and-chargepoint-as-charging-solution-providers-for-its-e-golf-2"
[5] "http://investing.businessweek.com/research/stocks/private/snapshot.asp?privcapId=84543228"
[6] "http://insideevs.com/volkswagen-selects-chargepoint-bosch-e-golf-charging/"
[7] "http://www.calcharge.org/2014/07/"
[8] "http://nl.anygator.com/search/volkswagen+winterbanden"
[9] "http://nl.anygator.com/search/winterbanden+actie+volkswagen+caddy"
为 phantomJS 编写一个简单的函数来检查网络流量
使用RSelenium 和phantomJS，我们还可以使用它来实时检查流量（目前仅在直接驱动phantomJS 时）。举个简单的例子，我们记录当前正在查看的网页请求和接收的呼叫，并将其存储在当前工作目录的“traffic.txt”中：
library(RSelenium)
pJS <- phantom()
remDr <- remoteDriver(browserName = "phantom")
remDr$open()
appUrl <- "http://spiderbook.com/company/17495/details?rel=300795"
psScript <- "var page = this;
var fs = require(\"fs\");
fs.write(\"traffic.txt\", 'WEBSITE CALLS\\n', 'w');
page.onResourceRequested = function(request) {
fs.write(\"traffic.txt\", 'Request: ' + request.url + '\\n', 'a');
};
page.onResourceReceived = function(response) {
fs.write(\"traffic.txt\", 'Receive: ' + response.url + '\\n', 'a');
};"
result <- remDr$phantomExecute(psScript)
remDr$navigate(appUrl)
urlTraffic <- readLines("traffic.txt")
> head(urlTraffic)
[1] "WEBSITE CALLS"
[2] "Request: http://spiderbook.com/company/17495/details?rel=300795"
[3] "Receive: http://spiderbook.com/company/17495/details?rel=300795"
[4] "Request: http://spiderbook.com/static/js/jquery-1.10.2.min.js"
[5] "Request: http://spiderbook.com/static/js/lib/jquery.dropkick-1.0.2.js"
[6] "Request: http://spiderbook.com/static/js/jquery.textfill.js"
> urlTraffic[grepl("Receive: http://spiderbook.com/company/details", urlTraffic)]
[1] "Receive: http://spiderbook.com/company/details/docs?rel=300795&docs_page=0"
[2] "Receive: http://spiderbook.com/company/details/docs?rel=300795&docs_page=0"
pJS$stop() # stop phantomJS
在这里我们可以看到收到的文件之一是"Receive: http://spiderbook.com/company/details/docs?rel=300795&docs_page=0"。
使用 phantomJS/ghostdriver 内置的 HAR 支持来检查流量
事实上phantomJS/ghostscript 会创建自己的HAR 文件，因此只需在我们驾驶phantomJS 时浏览页面，我们就可以访问所有请求/响应数据：
library(RSelenium)
pJS <- phantom()
remDr <- remoteDriver(browserName = "phantom")
remDr$open()
appUrl <- "http://spiderbook.com/company/17495/details?rel=300795"
remDr$navigate(appUrl)
harLogs <- remDr$log("har")[[1]]
harLogs <- fromJSON(harLogs$message)
# HAR contain alot of detail will just illustrate here accessing the data
requestURLs <- sapply(lapply(harLogs$log$entries, "[[", "request"), "[[","url")
requestHeaders <- lapply(lapply(harLogs$log$entries, "[[", "request"), "[[", "headers")
XHRIndex <- which(grepl("XMLHttpRequest", sapply(requestHeaders, sapply, "[[", "value")))
> harLogs$log$entries[XHRIndex][[1]]$request$url
[1] "http://spiderbook.com/company/details/docs?rel=300795&docs_page=0"
所以最后一个示例显示了查询phantomJS 生成的HAR 文件以查找XMLHttpRequest 请求，然后返回我们希望对应于我们在答案开头找到的json 文件的特定url。