【发布时间】：2015-02-28 08:17:12
【问题描述】：
我正在尝试使用web-page-replay 记录网站以进行实验。记录 HTTP 请求工作正常，但是当我尝试记录 HTTPS 请求时，我得到一个 SSLError。
在 OSX 上，我使用这个命令来记录网页
sudo ./replay.py --record archive.wpr
我已将系统 HTTP 代理设置为 localhost:80 并将 HTTPS 代理设置为 localhost:443
追溯——
2015-02-24 03:57:26,227 ERROR Client failed to make request
----------------------------------------
Exception happened during processing of request from ('127.0.0.1', 63226)
Traceback (most recent call last):
File "/usr/local/Cellar/python/2.7.9/Frameworks/Python.framework/Versions/2.7/lib/python2.7/SocketServer.py", line 599, in process_request_thread
self.finish_request(request, client_address)
File "/usr/local/Cellar/python/2.7.9/Frameworks/Python.framework/Versions/2.7/lib/python2.7/SocketServer.py", line 334, in finish_request
self.RequestHandlerClass(request, client_address, self)
File "/usr/local/Cellar/python/2.7.9/Frameworks/Python.framework/Versions/2.7/lib/python2.7/SocketServer.py", line 655, in __init__
self.handle()
File "/usr/local/Cellar/python/2.7.9/Frameworks/Python.framework/Versions/2.7/lib/python2.7/BaseHTTPServer.py", line 340, in handle
self.handle_one_request()
File "/Users/akshay/Projects/web-page-replay/httpproxy.py", line 162, in handle_one_request
self.raw_requestline = self.rfile.readline(65537)
File "/usr/local/Cellar/python/2.7.9/Frameworks/Python.framework/Versions/2.7/lib/python2.7/socket.py", line 476, in readline
data = self._sock.recv(self._rbufsize)
File "/usr/local/Cellar/python/2.7.9/Frameworks/Python.framework/Versions/2.7/lib/python2.7/ssl.py", line 714, in recv
return self.read(buflen)
File "/usr/local/Cellar/python/2.7.9/Frameworks/Python.framework/Versions/2.7/lib/python2.7/ssl.py", line 608, in read
v = self._sslobj.read(len or 1024)
SSLError: [SSL: HTTPS_PROXY_REQUEST] https proxy request (_ssl.c:1750)
----------------------------------------
更新 - 我尝试使用 telemetry 记录页面集，但在记录 https 页面时遇到 SSL 握手错误。
2015-03-01 11:12:04,929 WARNING HTTP server started on 127.0.0.1:50781
2015-03-01 11:12:04,930 WARNING HTTPS server started on 127.0.0.1:50782
2015-03-01 11:12:04,981 ERROR Client failed to make request
2015-03-01 11:12:04,981 ERROR Client failed to make request
Traceback (most recent call last):
File "/usr/local/Cellar/python/2.7.9/Frameworks/Python.framework/Versions/2.7/lib/python2.7/SocketServer.py", line 599, in process_request_thread
self.finish_request(request, client_address)
File "/usr/local/Cellar/python/2.7.9/Frameworks/Python.framework/Versions/2.7/lib/python2.7/SocketServer.py", line 334, in finish_request
self.RequestHandlerClass(request, client_address, self)
File "/usr/local/Cellar/python/2.7.9/Frameworks/Python.framework/Versions/2.7/lib/python2.7/SocketServer.py", line 655, in __init__
self.handle()
File "/usr/local/Cellar/python/2.7.9/Frameworks/Python.framework/Versions/2.7/lib/python2.7/BaseHTTPServer.py", line 340, in handle
self.handle_one_request()
File "/Users/akshay/Downloads/telemetry/src/third_party/webpagereplay/httpproxy.py", line 162, in handle_one_request
self.raw_requestline = self.rfile.readline(65537)
File "/usr/local/Cellar/python/2.7.9/Frameworks/Python.framework/Versions/2.7/lib/python2.7/socket.py", line 476, in readline
data = self._sock.recv(self._rbufsize)
File "/usr/local/Cellar/python/2.7.9/Frameworks/Python.framework/Versions/2.7/lib/python2.7/ssl.py", line 714, in recv
return self.read(buflen)
File "/usr/local/Cellar/python/2.7.9/Frameworks/Python.framework/Versions/2.7/lib/python2.7/ssl.py", line 608, in read
v = self._sslobj.read(len or 1024)
SSLError: [SSL: SSL_HANDSHAKE_FAILURE] ssl handshake failure (_ssl.c:1750)
2015-03-01 11:12:05,782 ERROR Client failed to make request
2015-03-01 11:12:07,611 WARNING Retrying fetch GET https://www.google.com/ [('accept-encoding', 'gzip, deflate, '), ('host', 'www.google.com')]: hostname '74.125.236.209' doesn't match 'www.google.com'
2015-03-01 11:12:08,844 WARNING Retrying fetch GET https://www.google.com/ [('accept-encoding', 'gzip, deflate, '), ('host', 'www.google.com')]: hostname '74.125.236.209' doesn't match 'www.google.com'
2015-03-01 11:12:09,539 WARNING Retrying fetch GET https://www.google.com/ [('accept-encoding', 'gzip, deflate, '), ('host', 'www.google.com')]: hostname '74.125.236.209' doesn't match 'www.google.com'
2015-03-01 11:12:10,146 CRITICAL Could not fetch GET https://www.google.com/ [('accept-encoding', 'gzip, deflate, '), ('host', 'www.google.com')]: hostname '74.125.236.209' doesn't match 'www.google.com'
2015-03-01 11:12:11,386 WARNING Retrying fetch GET https://www.google.com/favicon.ico [('accept-encoding', 'gzip, deflate, '), ('host', 'www.google.com')]: hostname '74.125.236.209' doesn't match 'www.google.com'
2015-03-01 11:12:13,428 WARNING Retrying fetch GET https://www.google.com/favicon.ico [('accept-encoding', 'gzip, deflate, '), ('host', 'www.google.com')]: hostname '74.125.236.209' doesn't match 'www.google.com'
2015-03-01 11:12:14,417 WARNING Retrying fetch GET https://www.google.com/favicon.ico [('accept-encoding', 'gzip, deflate, '), ('host', 'www.google.com')]: hostname '74.125.236.209' doesn't match 'www.google.com'
2015-03-01 11:12:18,349 CRITICAL Could not fetch GET https://www.google.com/favicon.ico [('accept-encoding', 'gzip, deflate, '), ('host', 'www.google.com')]: hostname '74.125.236.209' doesn't match 'www.google.com'
----------------------------------------
Exception happened during processing of request from ('127.0.0.1', 50785)
----------------------------------------
【问题讨论】：
-
看起来
web-page-replay正在以透明代理模式运行。如果您没有在 Chrome 中设置显式 HTTP/HTTPS 代理会怎样？ -
不，如果没有明确设置代理，它不会记录任何内容。此图说明了 WPR 如何记录和重放请求。
-
了解 - 我仔细查看了文档，它确实设置了本地代理。但是，通常 HTTP 和 HTTPs 代理服务位于同一个端口上，并且通过他们的文档，我没有发现任何证据表明他们希望通过 443 代理 HTTPs。此外，异常调用堆栈显示任何监听端口 443需要原始 SSL，并且很可能获得代理 CONNECT 方法。
-
当我在录制模式下运行服务器时，它会监听端口 53、80 和 443。所以我为端口 443 配置了 HTTPS 代理，否则 HTTPS 请求将被忽略。
-
我查看了源代码 - 我在任何地方都看不到处理 CONNECT 隧道的模块。当您在任何端口上的 HTTPs 上设置代理时，浏览器将发出一个 CONNECT，代理需要在 SSL 握手之前对其进行处理。它看起来不像
web-page-replay那样做——它只是将数据按原样转发到拒绝它的 SSL 层。最好联系作者。
标签： python-2.7 http ssl https chromium