抓取时从 VBA 访问 JavaScript 全局变量

Question

我正在用 VBA for Excel 编写一个检查页面的宏，我需要提取一个在 HTML 脚本中设置的变量。在宏中，我就这样创建了一个 IE 对象：

Set objIE = CreateObject("InternetExplorer.Application")

目标网站中的页面都有以下脚本，该脚本定义了一个名为digitalData.page.pageName 的变量。这是 HTML 的 sn-p：

<script>
var digitalData = '';
function init() {
digitalData = {
  "user": {
    "userLoginState": "guest",
    "userCountry": "",
    "userEmployeeName": "",
    "userBirthday": "",
    "userGender": "",
    "userState": "",
    "userID": "",
    "LRUserID": "",
    "userEmployeeID": "",
    "userDWID": "",
    "userSessionId": "BYTEzHFAdLrPoPPOlTPGWvlBjCx54jjEyB8="
  },
  "page": {
    "pageName": "en_us:plp:men:clothing:Casual Shirts",
    "pageType": "plp",
    "pageGender": "men",
    "pageLocale": "us",
    "pageRedirected": "no",
    "pageJSErrorCount": "3",
    "pageLevel1": "men",
    "pageLevel2": "men/clothing",
    "pageLevel3": "men/clothing/Casual Shirts",
    "pageLevel4": "men/clothing/Casual Shirts",
    "pageHierarchy": "men/clothing/Casual Shirts"
  },

如果我在 Chrome 中打开此站点的页面并检查它，我可以在控制台中键入变量名称，它会返回值，但我似乎无法使用 VBA 从 IE 访问变量，如下所示：

inspectLink(i, 1) = objIE.digitalData.page.pageName

在这种情况下，我想在inspectLink(i, 1) 中找到en_us:plp:men:clothing:Casual Shirts，但我却得到了Runtime error '438' Object doesn't support the property or method。

Dim inspectCat(4) As String
inspectCat(0) = "webcat=men"
inspectCat(1) = "webcat=women"
inspectCat(2) = "webcat=kids"
inspectCat(3) = "webcat=baby"
inspectCat(4) = "webcat=home"

Dim targetSearchCount as Integer
Dim failedSearchCount as Integer
targetSearchCount=0
failedSearchCount=0

REM New Code - DOES NOT WORK cannot access pageName this way
REM if digitalData.page.pageName has en_us: in it, then it's our target
REM if it has failedSearchResult in it, then report to web dev team
REM syntax might be objIE.Document.digitalData.page.pageName

REM inspectLink(i, 1) = objIE.digitalData.page.pageName
REM MsgBox inspectLink(i, 1)
REM if inStr(objIE.digitalData.page.pageName, "en_us:") then targetSearchCount=targetSearchCount+1 endif
REM if inStr(objIE.digitalData.page.pageName, "failed_Search_Result") then failedSearchCount=failedSearchCount+1 endif
REM End New Code

REM Begin Old Code - WORKS BUT "dublicate" MAY NOT BE RELIABLE OVER TIME
REM 
Set pageNameDubs = objIE.Document.GetElementsByClassName("page-Name-dublicate")
'MsgBox pageNameDubs(0).Value
For Each pageName In pageNameDubs
'    If InStr(pageName.innertext, "en_us:") > 0 Then   
        inspectLink(i, 1) = pageName.Value
'    End If
Next
REM End Old Code

Answer 1

这是一个有趣的。这个答案专门关于访问digitalData.page.pageName。在下面的代码中，VVV 和 ^^^ 之间的所有内容都基于已加载的文档执行此操作。您可以将该部分集成到现有代码中。

在开发者控制台中，这与digitalData.page.pageName 一样简单，或者等效地为document.defaultView.digitalData.page.pageName (source)。您可以在 Excel VBA 中获得 document.defaultView，但我不知道如何从该对象访问 JavaScript 全局变量。相反，我是通过 DOM 完成的。以下示例适用于我。

代码

首先，确保您已添加对 Microsoft Internet 控件和 Microsoft HTML 对象库的引用。

Option Explicit
Option Base 0

Public Sub GetResult()
    Dim objIE As SHDocVw.InternetExplorer
    Set objIE = New SHDocVw.InternetExplorer

    ' Load the page with the target data
    With objIE
        .navigate "http://cxw42.github.io/49290039.html?buster=1"
        ' Cache buster thanks to https://stackoverflow.com/questions/24851824/how-long-does-it-take-for-github-page-to-show-changes-after-changing-index-html#comment69647442_24871850
        ' by https://stackoverflow.com/users/185973/joel-glovier
        .Visible = True
    End With

    Do While objIE.Busy
        DoEvents
    Loop

    Dim doc As MSHTML.HTMLDocument
    Set doc = objIE.document

    ' VVVVVVVVVVVVVV

    Dim win As MSHTML.HTMLWindow2
    Set win = doc.defaultView

    ' Should be able to directly access win.digitalData, but I can't get that to work.
    ' Instead, access the data indirectly

    Dim uniqueid As String
    uniqueid = "id_" & Format(Now, "%yyyy%mm%dd%hh%nn%ss")
    Dim code As String
    code = "(function(){var x = document.createElement('p'); x.id='" & uniqueid & "'; x.innerText=digitalData.page.pageName; document.body.appendChild(x); })()"
        ' Copy digitalData.page.pageName into the DOM
    win.execScript code, "JavaScript"

    Dim pageName As String
    Dim node
    Set node = doc.getElementById(uniqueid)     ' Get the new DOM node
    pageName = node.innerText

    ' Clean up
    doc.getElementsByTagName("body").Item(0).RemoveChild node

    ' ^^^^^^^^^^^^^^        

    ' Now do whatever you want with pageName.
    Debug.Print pageName
End Sub

说明

魔法就在code 和win.execScript。 code 是一个 JavaScript oneliner，它创建一个新的 <p> 元素并将 digitalData.page.pageName 复制到其中。 win.execScript 在页面上下文中运行该 JavaScript，从而创建新节点。该节点有一个uniqueid（嗯，可能是唯一的），一旦它被创建，我们就可以使用它来查找它。然后我们可以从该段的innerText 中提取结果。

感谢

• document.defaultView
• Scraping using VBA
• Github Pages cache-busting
• The importance of early binding（阅读！）