SSL：尝试打开 Nav Canada 网站时出现 CERTIFICATE_VERIFY_FAILED

Question

我创建了一个脚本，它从 Nav Canada 网站上抓取最新的航空天气数据，地址如下；

https://flightplanning.navcanada.ca/cgi-bin/lab/lab.cgi?Langue=anglais&NoSession=NS_Inconnu&TypeBrief=L&Version=T&AerodromeId=CYOW&cw_metar=dcd_metar&fw_gfaCld=on&fw_gfaIcg=on&rwt_uprWindsFD180=on

我使用这些数据来计算一些变量，并使用树莓派将其显示在我自己的网络服务器上。经过几个月的无故障处理后，几天前我遇到了这个错误。我尝试将链接切换到不同的网站，看看它是否是请求模块的错误，但其他网站工作正常。有没有可能我被禁止以这种方式访问​​他们的网站？我仍然可以通过浏览器访问它。或者他们是我没有想到的另一个潜在问题？这是堆栈跟踪。

Traceback (most recent call last):
  File "C:\Program Files\Python36\lib\urllib\request.py", line 1318, in do_open
    encode_chunked=req.has_header('Transfer-encoding'))
  File "C:\Program Files\Python36\lib\http\client.py", line 1239, in request
    self._send_request(method, url, body, headers, encode_chunked)
  File "C:\Program Files\Python36\lib\http\client.py", line 1285, in _send_request
    self.endheaders(body, encode_chunked=encode_chunked)
  File "C:\Program Files\Python36\lib\http\client.py", line 1234, in endheaders
    self._send_output(message_body, encode_chunked=encode_chunked)
  File "C:\Program Files\Python36\lib\http\client.py", line 1026, in _send_output
    self.send(msg)
  File "C:\Program Files\Python36\lib\http\client.py", line 964, in send
    self.connect()
  File "C:\Program Files\Python36\lib\http\client.py", line 1400, in connect
    server_hostname=server_hostname)
  File "C:\Program Files\Python36\lib\ssl.py", line 401, in wrap_socket
    _context=self, _session=session)
  File "C:\Program Files\Python36\lib\ssl.py", line 808, in __init__
    self.do_handshake()
  File "C:\Program Files\Python36\lib\ssl.py", line 1061, in do_handshake
    self._sslobj.do_handshake()
  File "C:\Program Files\Python36\lib\ssl.py", line 683, in do_handshake
    self._sslobj.do_handshake()
ssl.SSLError: [SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed (_ssl.c:749)

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "C:/Users/Philip Naida/Google Drive/Projects/WeatherDataCollector/WeatherDataCollector v_5.0/WeatherDataCollector.py", line 15, in <module>
    source = urllib.request.urlopen('https://flightplanning.navcanada.ca/cgi-bin/CreePage.pl?Langue=anglais&NoSession=NS_Inconnu&Page=forecast-observation&TypeDoc=html').read()
  File "C:\Program Files\Python36\lib\urllib\request.py", line 223, in urlopen
    return opener.open(url, data, timeout)
  File "C:\Program Files\Python36\lib\urllib\request.py", line 526, in open
    response = self._open(req, data)
  File "C:\Program Files\Python36\lib\urllib\request.py", line 544, in _open
    '_open', req)
  File "C:\Program Files\Python36\lib\urllib\request.py", line 504, in _call_chain
    result = func(*args)
  File "C:\Program Files\Python36\lib\urllib\request.py", line 1361, in https_open
    context=self._context, check_hostname=self._check_hostname)
  File "C:\Program Files\Python36\lib\urllib\request.py", line 1320, in do_open
    raise URLError(err)
urllib.error.URLError: <urlopen error [SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed (_ssl.c:749)>

如果可以，你们中的任何人都可以尝试使用 urllib 访问此网页，看看会发生什么吗？任何帮助将不胜感激我想让我的服务器重新上线。

谢谢

Answer 1

查看此站点的report from SSLLabs，您会看到许多其他警告：

此服务器的证书链不完整。

如果您查看更多详细信息，您会发现服务器仅提供叶证书，但未能提供将信任链构建到本地根证书所需的中间证书。

桌面浏览器经常（但并非总是）通过尝试从其他地方自动获取丢失的证书来绕过此类错误配置的服务器。但其他应用程序（如 Python、Perl、Java... 应用程序）只是失败，因为如果没有缺少中间证书，则无法正确验证证书。

要解决此服务器问题，您需要自己提供中间证书。这可以通过以下请求来完成：

import requests
response = request.get('https://flightplanning.navcanada.ca', verify='mycerts.pem')

mycerts.pem 包含所有缺少的中间证书和您信任的根证书。你可以在https://pastebin.com/zi5BN2Vt找到这样一个文件来修复与这个特定服务器的连接

Answer 2

该网站使用 TLS 1.0（一种过时的协议）、RSA（一种过时的密钥交换）和带有 HMAC-SHA1（一种过时的密码）的 AES_128_CBC。如果你想继续使用它，我会推荐使用流行的requests 包。它允许您在发送请求时关闭服务器验证。

import requests

url = ('https://flightplanning.navcanada.ca/cgi-bin/lab/lab.cgi?Langue=anglais' 
       '&NoSession=NS_Inconnu&TypeBrief=L&Version=T&AerodromeId=CYOW&cw_metar='
       'dcd_metar&fw_gfaCld=on&fw_gfaIcg=on&rwt_uprWindsFD180=on')

res = request.get(url, verify=False)

# you will get the following warning
InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate 
verification is strongly advised. See: https://urllib3.readthedocs.io
/en/latest/advanced-usage.html#ssl-warnings    InsecureRequestWarning)

res
# returns:
<Response [200]>