来自请求 Python 库的 HTTP 请求中缺少主机标头答案

【问题标题】：Missing Host header in HTTP requests from the requests Python library来自请求 Python 库的 HTTP 请求中缺少主机标头
【发布时间】：2019-09-03 11:15:37
【问题描述】：

requests Python 库生成的 HTTP 请求消息中的HTTP/1.1 mandatory Host header field 在哪里？

import requests

response = requests.get("https://www.google.com/")
print(response.request.headers)

输出：

{'User-Agent': 'python-requests/2.22.0', 'Accept-Encoding': 'gzip, deflate', 'Accept': '*/*', 'Connection': 'keep-alive '}

【问题讨论】：

使用httpbin.org/get 获取服务器获取的所有标头。 requests.get("https://httpbin.org/get") print(response.content) 它似乎发送了Host，但它稍后没有显示。如果你在get(..., headers={'Host':'example.com'}) 中设置了错误的Host，那么它仍然可以正常工作——所以这个字段似乎不是那么强制性——然后它会在response.request.headers 中显示这个标题

标签： python http python-requests http-headers

【解决方案1】：

requests 默认情况下不会将 HOST 标头添加到请求中。如果未明确添加，则将决策委托给底层 http 模块。

查看http/client.py的这一部分：

（如果requests.get 中明确提供了'Host' 标头，则skip_host 是True）

    if self._http_vsn == 11:
        # Issue some standard headers for better HTTP/1.1 compliance

        if not skip_host:
            # this header is issued *only* for HTTP/1.1
            # connections. more specifically, this means it is
            # only issued when the client uses the new
            # HTTPConnection() class. backwards-compat clients
            # will be using HTTP/1.0 and those clients may be
            # issuing this header themselves. we should NOT issue
            # it twice; some web servers (such as Apache) barf
            # when they see two Host: headers

            # If we need a non-standard port,include it in the
            # header.  If the request is going through a proxy,
            # but the host of the actual URL, not the host of the
            # proxy.

            netloc = ''
            if url.startswith('http'):
                nil, netloc, nil, nil, nil = urlsplit(url)

            if netloc:
                try:
                    netloc_enc = netloc.encode("ascii")
                except UnicodeEncodeError:
                    netloc_enc = netloc.encode("idna")
                self.putheader('Host', netloc_enc)
            else:
                if self._tunnel_host:
                    host = self._tunnel_host
                    port = self._tunnel_port
                else:
                    host = self.host
                    port = self.port

                try:
                    host_enc = host.encode("ascii")
                except UnicodeEncodeError:
                    host_enc = host.encode("idna")

                # As per RFC 273, IPv6 address should be wrapped with []
                # when used as Host header

                if host.find(':') >= 0:
                    host_enc = b'[' + host_enc + b']'

                if port == self.default_port:
                    self.putheader('Host', host_enc)
                else:
                    host_enc = host_enc.decode("ascii")
                    self.putheader('Host', "%s:%s" % (host_enc, port))

因此，在检查 requests 发送到服务器的标头时，我们看不到 'Host' 标头。

如果我们向http://httpbin/get 发送请求并打印响应，我们可以看到Host 标头确实已发送。

import requests

response = requests.get("http://httpbin.org/get")
print('Response from httpbin/get')
print(response.json())
print()
print('response.request.headers')
print(response.request.headers)

输出

Response from httpbin/get
{'args': {}, 'headers': {'Accept': '*/*', 'Accept-Encoding': 'gzip, deflate', 
 'Host': 'httpbin.org', 'User-Agent': 'python-requests/2.20.0'},
 'origin': 'XXXXXX', 'url': 'https://httpbin.org/get'}

response.request.headers
{'User-Agent': 'python-requests/2.20.0', 'Accept-Encoding': 'gzip, deflate', 
 'Accept': '*/*', 'Connection': 'keep-alive'}

【讨论】：

谢谢，现在有意义了！