httplib.BadStatusLine: '' 与 Selenium 和 PhantomJS

Question

我在加载 URL 时遇到此错误，并且遇到一个奇怪的错误，我不知道如何解决。我的情况要求我必须使用 PhantomJS，因为我认为我不能在 AWS Lambda 上使用 Firefox 驱动程序，并且在我的抓取中我遇到了一个 Chromedriver 无法单击的按钮。

如果我将 PhantomJS 切换为 Chrome 或 Firefox，则 url 解析得很好。

使用 selenium==3.4.1 和 PhantomJS 2.1.1

user_agent = (
        "Mozilla/5.0 (Macintosh; Intel Mac OS X 10.12; rv:53.0) Gecko/20100101 Firefox/53.0")

dcap = dict(DesiredCapabilities.PHANTOMJS)
dcap["phantomjs.page.settings.userAgent"] = user_agent
dcap["phantomjs.page.settings.javascriptEnabled"] = True

browser = webdriver.PhantomJS(service_log_path=os.path.devnull, service_args=[
    '--ignore-ssl-errors=true'], desired_capabilities=dcap)

browser.set_window_size(1120, 550)
browser.get('https://drizly.com/session/new')



File "main.py", line 257, in <module>
    lambda_handler(None, None)
  File "main.py", line 103, in lambda_handler
    browser.get('https://drizly.com/session/new')
  File "/Users/aymon/Envs/drizly/lib/python2.7/site-packages/selenium/webdriver/remote/webdriver.py", line 264, in get
    self.execute(Command.GET, {'url': url})
  File "/Users/aymon/Envs/drizly/lib/python2.7/site-packages/selenium/webdriver/remote/webdriver.py", line 250, in execute
    response = self.command_executor.execute(driver_command, params)
  File "/Users/aymon/Envs/drizly/lib/python2.7/site-packages/selenium/webdriver/remote/remote_connection.py", line 464, in execute
    return self._request(command_info[0], url, body=data)
  File "/Users/aymon/Envs/drizly/lib/python2.7/site-packages/selenium/webdriver/remote/remote_connection.py", line 526, in _request
    resp = opener.open(request, timeout=self._timeout)
  File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/urllib2.py", line 431, in open
    response = self._open(req, data)
  File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/urllib2.py", line 449, in _open
    '_open', req)
  File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/urllib2.py", line 409, in _call_chain
    result = func(*args)
  File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/urllib2.py", line 1227, in http_open
    return self.do_open(httplib.HTTPConnection, req)
  File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/urllib2.py", line 1200, in do_open
    r = h.getresponse(buffering=True)
  File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/httplib.py", line 1132, in getresponse
    response.begin()
  File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/httplib.py", line 453, in begin
    version, status, reason = self._read_status()
  File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/httplib.py", line 417, in _read_status
    raise BadStatusLine(line)
httplib.BadStatusLine: ''

Answer 1

BadStatusLine 是HTTPException 的子类，如果a server responds with a HTTP status code that we don’t understand 则会引发该子类。你可能想抓住它，如下

#...
browser.set_window_size(1120, 550)
try:
    browser.get('https://drizly.com/session/new')
except httplib.BadStatusLine as bsl:
    print('[!!!] {message}'.format(bsl.message))
#...

请注意，一个好的做法是错误should never pass silently。因此使用print。