如何使用 VBA 在 HTML 页面上获取脚本生成的数据？

Question

我正在尝试使用 VBA 解析 Website。我的目标是提取公司的数据（名称、网站、位置、投资者、投资日期、投资金额等）并将其存储在 Excel 表中。我提取 HTML 源代码的功能如下（至少在我的计算机上有效......）。我使用后期绑定以获得更好的可移植性。

Private Sub getHTMLFromURL()

    Dim objHTTP As Object
    Dim html As Object
    Dim implTmp As Object

    Set objHTTP = VBA.CreateObject("MSXML2.XMLHTTP")
    Set html = CreateObject("htmlfile")

    on error GoTo endProgram

    With objHTTP
        .Open "GET", URLStr, False
        .send

        If .READYSTATE = 4 And .Status = 200 Then
            html.Open
            html.write .responseText
            html.Close
        Else
            Debug.Print "Error" & vbNewLine & "Ready state: " & .READYSTATE & _
            vbNewLine & "HTTP request status: " & .Status
            GoTo endProgram
        End If

    End With

endProgram:
    Set html = Nothing
    Set objHTTP = Nothing
    If Err <> 0 Then
        Debug.Print "Error in getHTMLFromURL " & Err.Number & " - " & Err.Description
    End If

End Sub

我的问题是 HTML 是一个巨大的 SCRIPT 标记，其中隐藏了数据。例如，我猜公司的总部位置在这行代码中：

{"key":"hqLocations","name":"Location","sortable":false,"getText":"function getText(c) {\n    return getHQCity(c.hqLocations);\n  }"}

我谦虚地承认，我完全不知道如何获取这些数据。我在多个论坛中搜索了很多，但没有找到合适的答案。我试图适应this method，但没有成功。因此，我有几个与我的问题相关的问题：

• 使用 MSXML2.XMLHTTP 是最好的方法吗？
• 我是否必须对每个变量进行钓鱼，或者有没有办法直接解释脚本以获得包含所有数据的 HTML（这样会更容易）？
• 否则，如何提取所有数据（单个数据和数组）？

非常感谢

Answer 1

试试下面的脚本。当你运行它时，你应该得到你在帖子中请求的所需数据：

Sub Fetch_Data()
    Dim IE As New InternetExplorer, HTML As HTMLDocument
    Dim posts As Object, post As Object, hdata As Object
    Dim elem As Object, trow As Object

    With IE
        .Visible = False
        .navigate "http://app.startupeuropeclub.eu/companies/pld_space"
        While .Busy = True Or .readyState < 4: DoEvents: Wend
        Set HTML = .document
    End With

    ''the following line is let the scraper wait until the data is completely loaded
    Do: Set hdata = HTML.getElementsByClassName("field"): DoEvents: Loop Until hdata.Length > 1

    For Each post In hdata
        With post.getElementsByClassName("title")
            If .Length Then R = R + 1: Cells(R, 1) = .Item(0).innerText
        End With
        With post.getElementsByClassName("description")
            If .Length Then Cells(R, 2) = .Item(0).innerText
        End With
    Next post

    ''avoiding hardcoded delay and wait until the data is completely loaded
    Do: Set posts = HTML.querySelector(".card-content.with-padding table.simple-table"): DoEvents: Loop While posts Is Nothing

    For Each elem In posts.Rows
        For Each trow In elem.Cells
            C = C + 1: Cells(I + 10, C) = trow.innerText
        Next trow
        C = 0
        I = I + 1
    Next elem
    IE.Quit
End Sub

添加到库的参考：

Microsoft Internet Controls
Microsoft HTML Object Library