Powershell Web Scraping SSL/TLS 问题

Question

我想在服务器上运行网络抓取脚本。

当前脚本收集指定页面的html。

$url = "http://websms"
 [net.httpWebRequest] $request = [net.webRequest]::create($url)
 [net.httpWebResponse] $response = $request.getResponse()
 $responseStream = $response.getResponseStream()
 $sr = new-object IO.StreamReader($responseStream)
 $result = $sr.ReadToEnd()

 $result

这在典型的网页上运行良好。但是我想在服务器管理页面上运行它，这当然需要登录。

在尝试登录之前，我想我会尝试抓取服务器的登录页面。运行上面的脚本，我得到以下结果。

   Exception calling "GetResponse" with "0" argument(s): "The underlying connection was closed: Could not establish trust relationship for the SSL/TLS secure channel."
At C:\temp\web3.ps1:3 char:56
+  [net.httpWebResponse] $response = $request.getResponse <<<< ()
    + CategoryInfo          : NotSpecified: (:) [], MethodInvocationException
    + FullyQualifiedErrorId : DotNetMethodException

关于如何解决此问题的任何想法，或者您是否可以为我指出不同的方向，以便我可以从服务器的管理 html 页面中抓取元素。

谢谢各位！

Answer 1

这一行将忽略 SSL 证书错误：

[System.Net.ServicePointManager]::ServerCertificateValidationCallback = {$true}

执行此操作后，将忽略有关自签名不受信任证书、名称不匹配或过期的错误。

Answer 2

使用this brilliant answer:

public bool AcceptAllCertifications(object sender, System.Security.Cryptography.X509Certificates.X509Certificate certification, System.Security.Cryptography.X509Certificates.X509Chain chain, System.Net.Security.SslPolicyErrors sslPolicyErrors)
{
    return true;
}
ServicePointManager.ServerCertificateValidationCallback = new System.Net.Security.RemoteCertificateValidationCallback(AcceptAllCertifications);