【问题标题】:Can't parse name from a webpage using POST request无法使用 POST 请求从网页中解析名称
【发布时间】:2017-08-14 04:40:03
【问题描述】:

我在 vba 中编写了一个宏来使用 POST 请求从网站获取“名称”。要到达目标页面,必须发送两次 POST 请求。首先,打开一个页面,就像下面的第一张图片一样。单击起始页上的“按地址搜索”按钮后,它会转到另一个页面,其中需要填写两个框,如下图 2 所示。一个用于街道编号,另一个用于街道名称。填写完表单后单击搜索按钮后,它会引导到目标页面,其中包含我所追求的信息。我在脚本中使用 msgbox 对其进行了测试,以确保我在正确的页面上。我肯定在那个页面上,我可以看到那个页面的标题是“哈里斯县评估区”。但是,我无法解析该页面中的任何内容。我在那个页面上找到了这个名字“LARA PEDRO A & MARIA G”。

这是我正在尝试的宏:

Sub httpPost()

    Dim http As New XMLHTTP60, html As New HTMLDocument
    Dim rec As HTMLHtmlElement
    Dim ArgStr As String, ArgStr_ano As String

    ArgStr = "search=addr"
    ArgStr_ano = "TaxYear=2017&stnum=15535&stname=CAMPDEN+HILL+RD"

    With http
        .Open "POST", "https://public.hcad.org/records/QuickSearch.asp", False
        .setRequestHeader "Content-type", "application/x-www-form-urlencoded"
        .setRequestHeader "User-Agent", "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/60.0.3112.90 Safari/537.36"
        .setRequestHeader "Referer", "https://public.hcad.org/records/quicksearch.asp"
        .send ArgStr
    End With

    With http
        .Open "POST", "https://public.hcad.org/records/QuickRecord.asp", False
        .setRequestHeader "Content-type", "application/x-www-form-urlencoded"
        .setRequestHeader "User-Agent", "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/60.0.3112.90 Safari/537.36"
        .setRequestHeader "Referer", "https://public.hcad.org/records/quicksearch.asp"
        .send ArgStr_ano
        html.body.innerHTML = .responseText
    End With

    MsgBox http.responseText
End Sub

搜索:

Street No: 15535
Street Name: CAMPDEN HILL RD

这些是可以到达目标页面的两个页面的图像:

"https://www.dropbox.com/s/e9on9zwqzmcboze/1Untitled.jpg?dl=0" "https://www.dropbox.com/s/0lchpde8uq63jps/pics.jpg?dl=0"

我以某种方式使用 chrome 开发人员工具捕获了 url,并在下面的代码中使用了该 url,我得到了我想要的结果。但是,我的“POST”请求有什么问题?为什么我不能使用上述方法得到相同的结果?为了您的考虑,这里有另一段代码,可以使用从 chrome 开发工具收集的 url 来获取结果,我通过发送 post 请求两次得到结果:

Sub Web_Data()
    Dim http As New XMLHTTP60, html As New HTMLDocument
    Dim post As Object

    With http
        .Open "GET", "https://public.hcad.org/records/details.asp?crypt=%94%9A%B0%94%BFg%85%8D%83%82og%8El%87tXvXQJXJzDTpHjEyr%D4%BE%C2%AF%AE%AA%9Fpk%88%5Do%5B%B8%96%A3%C0q%5E&bld=1&tab=", False
        .send
        html.body.innerHTML = .responseText
    End With

    For Each post In html.getElementsByClassName("data")(2).getElementsByTagName("th")
        i = i + 1: Cells(i, 1) = post.innerText
    Next post
End Sub

结果是:

LARA PEDRO A & MARIA G
15531 CAMPDEN HILL RD
HOUSTON TX 77053-3302

【问题讨论】:

标签: vba web-scraping http-post


【解决方案1】:

终于自己解决了。这是工作代码:

Sub httpPost()

    Dim http As New WinHttp.WinHttpRequest, html As New HTMLDocument
    Dim post As Object
    Dim ArgStr As String, ArgStr_ano As String

    ArgStr = "search=addr"
    ArgStr_ano = "TaxYear=2017&stnum=15535&stname=CAMPDEN+HILL+RD"

    With http
        .Option(6) = True
        .Open "POST", "https://public.hcad.org/records/QuickSearch.asp", False
        .setRequestHeader "Content-type", "application/x-www-form-urlencoded"
        .setRequestHeader "User-Agent", "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/60.0.3112.90 Safari/537.36"
        .setRequestHeader "Referer", "https://public.hcad.org/records/quicksearch.asp"
        .send ArgStr
    End With

    With http
        .Option(6) = True
        .Open "POST", "https://public.hcad.org/records/QuickRecord.asp", False
        .setRequestHeader "Content-type", "application/x-www-form-urlencoded"
        .setRequestHeader "User-Agent", "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/60.0.3112.90 Safari/537.36"
        .setRequestHeader "Referer", "https://public.hcad.org/records/quicksearch.asp"
        .send ArgStr_ano
        html.body.innerHTML = .responseText
    End With

    For Each post In html.getElementsByClassName("data")(2).getElementsByTagName("th")
        i = i + 1: Cells(i, 1) = post.innerText
    Next post

End Sub

【讨论】:

    猜你喜欢
    • 1970-01-01
    • 2021-07-16
    • 2023-04-01
    • 2020-02-06
    • 2020-05-26
    • 1970-01-01
    • 1970-01-01
    • 1970-01-01
    • 1970-01-01
    相关资源
    最近更新 更多