带有 HTTPS 的 WebClient.DownloadString“身份验证或解密失败”

Question

我正在使用 Windows 10 上的 Unity Engine(2017.2.0f3) 创建游戏，我正在尝试从网页获取 HTML。据我所知，Unity使用的是“Mono”C#编译器和运行时v2.0（根据mono --version在@ 987654329@），还有v5.0.1（根据Unity\Editor\Data\MonoBleedingEdge\bin中运行mono --version的结果）。我目前正在使用 HTML Agility Pack 进行解析。尝试访问 HTTP 安全网站时，例如 dictionary.com，一切正常，但无论我做什么，我都无法访问 HTTPS 安全网站，例如作为urbandictionary.com（编辑：显然，问题是特定于该站点，因为使用mozroots 导入证书后，可以访问github.com），并且总是收到异常：TlsException: The authentication or decryption has failed. 我知道 HTML Agility Pack 不支持HTTPS，我的代码是按照this answer 写的。我已经尝试过here 和here 描述的解决方案，但无济于事。我试过运行mozroots，我位于PathTo/Unity/Editor/Data/MonoBleedingEdge/lib/mono/4.5/mozroots，带有--import、--sync和--quiet标志，但抛出了同样的异常（如果这确实有效，我对此表示怀疑它的可移植性，但也许我可以按照 cmets 的建议实现我自己的 mozroots 版本）。另外，我被告知 Unity 使用的 Mono 版本不支持 TLS 1.2，这是有问题的。如果没有解决这个问题，是否有任何解决方法？

注意：为清楚起见进行了删节

class MyWebClient : WebClient
{
    protected override WebRequest GetWebRequest(Uri address)
    {
        HttpWebRequest request = base.GetWebRequest(address) as HttpWebRequest;
        request.AutomaticDecompression = DecompressionMethods.Deflate | DecompressionMethods.GZip;
        return request;
    }
}

class GetHtml
{
    public static void Get(string url)
    {
        string documentString = new MyWebClient().DownloadString(url);
        ...
    }
}

class CallingClass
{
     public void CallingMethod()
     {
         ServicePointManager.ServerCertificateValidationCallback += (sender, certificate, chain, sslPolicyErrors) => true;
         //With or without the above line ^, the same TlsException is thrown. 
         GetHtml.Get("https://www.urbandictionary.com/define.php?term=example");
         //HTTP works properly, but HTTPS, as in this example, does not.
     }
}

日志如下：

TlsException: The authentication or decryption has failed.
Mono.Security.Protocol.Tls.RecordProtocol.ProcessAlert (AlertLevel alertLevel, AlertDescription alertDesc)
Mono.Security.Protocol.Tls.RecordProtocol.InternalReceiveRecordCallback (IAsyncResult asyncResult)
Rethrow as IOException: The authentication or decryption has failed.
Mono.Security.Protocol.Tls.SslStreamBase.AsyncHandshakeCallback (IAsyncResult asyncResult)
Rethrow as WebException: Error getting response stream (Write: The authentication or decryption has failed.): SendFailure
System.Net.HttpWebRequest.EndGetResponse (IAsyncResult asyncResult)
System.Net.HttpWebRequest.GetResponse ()
System.Net.WebClient.GetWebResponse (System.Net.WebRequest request)
System.Net.WebClient.DownloadDataCore (System.Uri address, System.Object userToken)

编辑：我已尝试更新运行时版本。唯一的选择是在 Project Settings->Player 菜单中从 .Net 3.5 更改为 4.6，这是实验性的。运行代码时仍然出现异常，但输出日志有点不同。

TlsException: The authentication or decryption has failed.
Mono.Security.Protocol.Tls.RecordProtocol.EndReceiveRecord (System.IAsyncResult asyncResult) (at <eb1224ae7b184cd09343d47f8a05481b>:0)
Mono.Security.Protocol.Tls.SslClientStream.SafeEndReceiveRecord (System.IAsyncResult ar, System.Boolean ignoreEmpty) (at <eb1224ae7b184cd09343d47f8a05481b>:0)
Mono.Security.Protocol.Tls.SslClientStream.NegotiateAsyncWorker (System.IAsyncResult result) (at <eb1224ae7b184cd09343d47f8a05481b>:0)
Rethrow as IOException: The authentication or decryption has failed.
Mono.Security.Protocol.Tls.SslClientStream.EndNegotiateHandshake (System.IAsyncResult result) (at <eb1224ae7b184cd09343d47f8a05481b>:0)
Mono.Security.Protocol.Tls.SslStreamBase.AsyncHandshakeCallback (System.IAsyncResult asyncResult) (at <eb1224ae7b184cd09343d47f8a05481b>:0)
Rethrow as IOException: The authentication or decryption has failed.
Mono.Security.Protocol.Tls.SslStreamBase.EndRead (System.IAsyncResult asyncResult) (at <eb1224ae7b184cd09343d47f8a05481b>:0)
Mono.Net.Security.Private.LegacySslStream.EndAuthenticateAsClient (System.IAsyncResult asyncResult) (at <344dc4d3f1ad41809df78607b6121a41>:0)
Mono.Net.Security.Private.LegacySslStream.AuthenticateAsClient (System.String targetHost, System.Security.Cryptography.X509Certificates.X509CertificateCollection clientCertificates, System.Security.Authentication.SslProtocols enabledSslProtocols, System.Boolean checkCertificateRevocation) (at <344dc4d3f1ad41809df78607b6121a41>:0)
Mono.Net.Security.MonoTlsStream.CreateStream (System.Byte[] buffer) (at <344dc4d3f1ad41809df78607b6121a41>:0)
System.Net.WebConnection.CreateStream (System.Net.HttpWebRequest request) (at <344dc4d3f1ad41809df78607b6121a41>:0)
Rethrow as WebException: Error: SecureChannelFailure (The authentication or decryption has failed.)
System.Net.WebClient.DownloadDataInternal (System.Uri address, System.Net.WebRequest& request) (at <344dc4d3f1ad41809df78607b6121a41>:0)
System.Net.WebClient.DownloadString (System.Uri address) (at <344dc4d3f1ad41809df78607b6121a41>:0)
System.Net.WebClient.DownloadString (System.String address) (at <344dc4d3f1ad41809df78607b6121a41>:0)

Answer 1

这个问题是由 HTTP 协议引起的。如果服务器使用安全的HTTP（实际上是HTTPS），可能会在底层协议无法处理X509 Certificate时产生错误。

尝试改变这一行：

HttpWebRequest request = base.GetWebRequest(address) as HttpWebRequest;

到：

HttpWebRequest request = base.GetWebRequest(address) as HttpWebRequest; 
request.ServerCertificateValidationCallback += (sender, certificate, chain, sslPolicyErrors) => true;

或者，或者，将验证回调应用到全局过滤器，使用：

ServicePointManager.ServerCertificateValidationCallback += (sender, certificate, chain, sslPolicyErrors) => true;

在您的代码中创建任何HttpWebRequest 对象之前。

如果您想更仔细地处理证书验证，请执行如下所示的回调：https://msdn.microsoft.com/en-us/library/office/dd633677

这应该可以解决您的问题。

Answer 2

经过严格的研究发现，Unity(2017.2.0f3) 当前使用的 Mono 版本不支持 TLS 1.2 ，正如开发人员here 所解释的那样。此外，对异常日志的仔细检查表明，在内部，正在使用遗留后端：Mono.Net.Security.Private.LegacySslStream 而不是Mono.Net.Security.Private.SslStream。截至目前，唯一的解决方案将涉及第三方库或解决方法。

感谢@Evk 为我指明了正确的方向。