Python 请求不适用于网站 API，但适用于浏览器（chrome）

Question

Python 请求不适用于网站 API，但适用于浏览器 (chrome)。它以前可以工作，但从过去的 2 个月开始，它就不能工作了。我使用的 api 是 yts.mx api。

下面我附上代码sn-p：

import requests
a = requests.get('https://yts.mx/api/v2/list_movies.json')

这是我得到的错误：

Traceback (most recent call last):
  File "D:\Programs\Python\Python39\lib\site-packages\urllib3\connectionpool.py", line 670, in urlopen
    httplib_response = self._make_request(
  File "D:\Programs\Python\Python39\lib\site-packages\urllib3\connectionpool.py", line 426, in _make_request
    six.raise_from(e, None)
  File "<string>", line 3, in raise_from
  File "D:\Programs\Python\Python39\lib\site-packages\urllib3\connectionpool.py", line 421, in _make_request
    httplib_response = conn.getresponse()
  File "D:\Programs\Python\Python39\lib\http\client.py", line 1347, in getresponse
    response.begin()
  File "D:\Programs\Python\Python39\lib\http\client.py", line 307, in begin
    version, status, reason = self._read_status()
  File "D:\Programs\Python\Python39\lib\http\client.py", line 268, in _read_status
    line = str(self.fp.readline(_MAXLINE + 1), "iso-8859-1")
  File "D:\Programs\Python\Python39\lib\socket.py", line 704, in readinto
    return self._sock.recv_into(b)
  File "D:\Programs\Python\Python39\lib\ssl.py", line 1241, in recv_into
    return self.read(nbytes, buffer)
  File "D:\Programs\Python\Python39\lib\ssl.py", line 1099, in read
    return self._sslobj.read(len, buffer)
TimeoutError: [WinError 10060] A connection attempt failed because the connected party did not properly respond after a period of time, or established connection failed because connected host has failed to respond

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "D:\Programs\Python\Python39\lib\site-packages\requests\adapters.py", line 439, in send
    resp = conn.urlopen(
  File "D:\Programs\Python\Python39\lib\site-packages\urllib3\connectionpool.py", line 726, in urlopen
    retries = retries.increment(
  File "D:\Programs\Python\Python39\lib\site-packages\urllib3\util\retry.py", line 410, in increment
    raise six.reraise(type(error), error, _stacktrace)
  File "D:\Programs\Python\Python39\lib\site-packages\urllib3\packages\six.py", line 734, in reraise
    raise value.with_traceback(tb)
  File "D:\Programs\Python\Python39\lib\site-packages\urllib3\connectionpool.py", line 670, in urlopen
    httplib_response = self._make_request(
  File "D:\Programs\Python\Python39\lib\site-packages\urllib3\connectionpool.py", line 426, in _make_request
    six.raise_from(e, None)
  File "<string>", line 3, in raise_from
  File "D:\Programs\Python\Python39\lib\site-packages\urllib3\connectionpool.py", line 421, in _make_request
    httplib_response = conn.getresponse()
  File "D:\Programs\Python\Python39\lib\http\client.py", line 1347, in getresponse
    response.begin()
  File "D:\Programs\Python\Python39\lib\http\client.py", line 307, in begin
    version, status, reason = self._read_status()
  File "D:\Programs\Python\Python39\lib\http\client.py", line 268, in _read_status
    line = str(self.fp.readline(_MAXLINE + 1), "iso-8859-1")
  File "D:\Programs\Python\Python39\lib\socket.py", line 704, in readinto
    return self._sock.recv_into(b)
  File "D:\Programs\Python\Python39\lib\ssl.py", line 1241, in recv_into
    return self.read(nbytes, buffer)
  File "D:\Programs\Python\Python39\lib\ssl.py", line 1099, in read
    return self._sslobj.read(len, buffer)
urllib3.exceptions.ProtocolError: ('Connection aborted.', TimeoutError(10060, 'A connection attempt failed because the connected party did not properly respond after a period of time, or established connection failed because connected host has failed to respond', None, 10060, None))

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "D:\Programs\Python\Python39\test.py", line 8, in <module>
    a = r.get('https://yts.mx/api/v2/list_movies.json')
  File "D:\Programs\Python\Python39\lib\site-packages\requests\api.py", line 76, in get
    return request('get', url, params=params, **kwargs)
  File "D:\Programs\Python\Python39\lib\site-packages\requests\api.py", line 61, in request
    return session.request(method=method, url=url, **kwargs)
  File "D:\Programs\Python\Python39\lib\site-packages\requests\sessions.py", line 530, in request
    resp = self.send(prep, **send_kwargs)
  File "D:\Programs\Python\Python39\lib\site-packages\requests\sessions.py", line 643, in send
    r = adapter.send(request, **kwargs)
  File "D:\Programs\Python\Python39\lib\site-packages\requests\adapters.py", line 498, in send
    raise ConnectionError(err, request=request)
requests.exceptions.ConnectionError: ('Connection aborted.', TimeoutError(10060, 'A connection attempt failed because the connected party did not properly respond after a period of time, or established connection failed because connected host has failed to respond', None, 10060, None))

有时由于现有连接而出现强制关闭等错误，这是跟踪：

Traceback (most recent call last):
  File "D:\Programs\Python\Python39\lib\site-packages\urllib3\connectionpool.py", line 670, in urlopen
    httplib_response = self._make_request(
  File "D:\Programs\Python\Python39\lib\site-packages\urllib3\connectionpool.py", line 381, in _make_request
    self._validate_conn(conn)
  File "D:\Programs\Python\Python39\lib\site-packages\urllib3\connectionpool.py", line 978, in _validate_conn
    conn.connect()
  File "D:\Programs\Python\Python39\lib\site-packages\urllib3\connection.py", line 362, in connect
    self.sock = ssl_wrap_socket(
  File "D:\Programs\Python\Python39\lib\site-packages\urllib3\util\ssl_.py", line 386, in ssl_wrap_socket
    return context.wrap_socket(sock, server_hostname=server_hostname)
  File "D:\Programs\Python\Python39\lib\ssl.py", line 500, in wrap_socket
    return self.sslsocket_class._create(
  File "D:\Programs\Python\Python39\lib\ssl.py", line 1040, in _create
    self.do_handshake()
  File "D:\Programs\Python\Python39\lib\ssl.py", line 1309, in do_handshake
    self._sslobj.do_handshake()
ConnectionResetError: [WinError 10054] An existing connection was forcibly closed by the remote host

这曾经在一个月左右之前完美运行。尝试了网上看到的各种方法。

更新

原来问题出在 IP 上。当我使用 VPN 连接时，它可以完美运行。但是，我不需要 VPN 就可以从我的网络浏览器访问它。知道为什么会这样吗？ 有没有办法绕过这个？

Answer 1

既然你提到浏览器请求通过但 python 请求没有通过，我想到了网站本身阻止了非浏览器请求。我发现了这个问题here，它解决了这个问题。解决方案是在 python 请求中添加自定义标头，使其看起来像浏览器请求：

import requests
headers = {'User-Agent': 'Mozilla/5.0 (Windows NT 6.1; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/56.0.2924.76 Safari/537.36'} # This is chrome, you can set whatever browser you like
url = 'https://yts.mx/api/v2/list_movies.json'
a = requests.get(url,headers)
print(a.content)

根据您的请求，然后将自定义标头作为 get-request 中的第二个参数发送。我尝试了上面的脚本，它对我有用，而且你的原始脚本也适用于我的连接。