来自 SEC Edgar 网站的 R XML href 抓取

Question

我检查了之前的类似问题 - 没有运气......似乎可以让 readHTMLTable 阅读 Edgar 网页。我正在尝试阅读此网址：

https://www.sec.gov/cgi-bin/browse-edgar?action=getcompany&CIK=AAPL&type=10-Q&dateb=&owner=exclude&count=100

...并将“文档”按钮下的所有 href 链接放入字符向量中。

“文档”链接位于表格中 - 来自 Firefox 检查工具的第一个“文档”href 链接如下所示：

<div id="seriesDiv" style="margin-top: 0px;">

    <table class="tableFile2" summary="Results">
        <tbody>
            <tr></tr>
            <tr>
                <td nowrap="nowrap"></td>
                <td nowrap="nowrap">
                    <a id="documentsbutton" href="/Archives/edgar/data/320193/000119312516559625/0001193125-16-559625-index.htm">

                         Documents

所以我想把href链接变成一个字符向量，以备后用。

问题 - XML 库给我带来了麻烦，htmltab 库函数似乎由于某种原因在我的 R 实例中没有得到识别。

这是我的代码：

library(XML)
EDGARURL <- "https://www.sec.gov/cgi-bin/browse-edgar?action=getcompany&CIK=AAPL&type=10-Q&dateb=&owner=exclude&count=100"
EDGARHREFtables <- readHTMLTable(EDGARURL, as.data.frame = TRUE)

这会导致以下错误：

    Warning message:
XML content does not seem to be XML: 'https://www.sec.gov/cgi-bin/browse-edgar?action=getcompany&CIK=AAPL&type=10-Q&dateb=&owner=exclude&count=100'

我错过了什么？ XML 图书馆的 readHTMLTable 会在这方面工作吗？如果是这样，您如何提取每个文档的 href 标记？

Answer 1

对于简单的工作，rvest 包要容易得多：

library(rvest)

url <- 'https://www.sec.gov/cgi-bin/browse-edgar?action=getcompany&CIK=AAPL&type=10-Q&dateb=&owner=exclude&count=100'

        # pull HTML from page  
url %>% read_html() %>%
    # get tags with a certain CSS selector
    html_nodes('#documentsbutton') %>%
    # get the href attribute from each node
    html_attr('href')

# [1] "/Archives/edgar/data/320193/000119312516559625/0001193125-16-559625-index.htm"
# [2] "/Archives/edgar/data/320193/000119312516439878/0001193125-16-439878-index.htm"
# [3] "/Archives/edgar/data/320193/000119312515259935/0001193125-15-259935-index.htm"
# [4] "/Archives/edgar/data/320193/000119312515153166/0001193125-15-153166-index.htm"
# [5] "/Archives/edgar/data/320193/000119312515023697/0001193125-15-023697-index.htm"
# [6] "/Archives/edgar/data/320193/000119312514277160/0001193125-14-277160-index.htm"
# [7] "/Archives/edgar/data/320193/000119312514157311/0001193125-14-157311-index.htm"
# [8] "/Archives/edgar/data/320193/000119312514024487/0001193125-14-024487-index.htm"
# [9] "/Archives/edgar/data/320193/000119312513300670/0001193125-13-300670-index.htm"
# [10] "/Archives/edgar/data/320193/000119312513168288/0001193125-13-168288-index.htm"
# ...