使用 Python Requests 库将 unicode 字符串发布到 Web 服务答案

【问题标题】：Post unicode string to web service using Python Requests library使用 Python Requests 库将 unicode 字符串发布到 Web 服务
【发布时间】：2016-04-09 15:49:28
【问题描述】：

我正在尝试使用 requests 库将包含精美 unicode 符号的文本 sn-p 发布到 Web 服务。我正在使用 Python 3.5。

text = "Två dagar kvar????????"
r = requests.post("http://json-tagger.herokuapp.com/tag", data=text)
print(r.json()

我收到一个 UnicodeEncodeError，但我无法弄清楚我做错了什么，请求的文档仅从我看到的内容中讨论 GET 请求中的 unicode。

    UnicodeEncodeError                        Traceback (most recent call last)
<ipython-input-125-3ebcae3d7918> in <module>()
     19         print("cleaned : " + line)
     20 
---> 21         r = requests.post("http://json-tagger.herokuapp.com/tag", data=line)
     22         sentences = r.json()['sentences']
     23         for sentence in sentences:

//anaconda/lib/python3.4/site-packages/requests/api.py in post(url, data, json, **kwargs)
    105     """
    106 
--> 107     return request('post', url, data=data, json=json, **kwargs)
    108 
    109 

//anaconda/lib/python3.4/site-packages/requests/api.py in request(method, url, **kwargs)
     51     # cases, and look like a memory leak in others.
     52     with sessions.Session() as session:
---> 53         return session.request(method=method, url=url, **kwargs)
     54 
     55 

//anaconda/lib/python3.4/site-packages/requests/sessions.py in request(self, method, url, params, data, headers, cookies, files, auth,     timeout, allow_redirects, proxies, hooks, stream, verify, cert, json)
    466         }
    467         send_kwargs.update(settings)
--> 468         resp = self.send(prep, **send_kwargs)
    469 
    470         return resp

//anaconda/lib/python3.4/site-packages/requests/sessions.py in send(self, request, **kwargs)
    574 
    575         # Send the request
--> 576         r = adapter.send(request, **kwargs)
    577 
    578         # Total elapsed time of the request (approximately)

//anaconda/lib/python3.4/site-packages/requests/adapters.py in send(self, request, stream, timeout, verify, cert, proxies)
    374                     decode_content=False,
    375                     retries=self.max_retries,
--> 376                     timeout=timeout
    377                 )
    378 

//anaconda/lib/python3.4/site-packages/requests/packages/urllib3/connectionpool.py in urlopen(self, method, url, body, headers, retries,     redirect, assert_same_host, timeout, pool_timeout, release_conn, **response_kw)
    557             httplib_response = self._make_request(conn, method, url,
    558                                                   timeout=timeout_obj,
--> 559                                                   body=body, headers=headers)
    560 
    561             # If we're going to release the connection in ``finally:``, then

//anaconda/lib/python3.4/site-packages/requests/packages/urllib3/connectionpool.py in _make_request(self, conn, method, url, timeout,     **httplib_request_kw)
    351         # conn.request() calls httplib.*.request, not the method in
    352         # urllib3.request. It also calls makefile (recv) on the socket.
--> 353         conn.request(method, url, **httplib_request_kw)
    354 
    355         # Reset the timeout for the recv() on the socket

//anaconda/lib/python3.4/http/client.py in request(self, method, url, body, headers)
   1086     def request(self, method, url, body=None, headers={}):
   1087         """Send a complete request to the server."""
-> 1088         self._send_request(method, url, body, headers)
   1089 
   1090     def _set_content_length(self, body):

//anaconda/lib/python3.4/http/client.py in _send_request(self, method, url, body, headers)
   1123             # RFC 2616 Section 3.7.1 says that text default has a
   1124             # default charset of iso-8859-1.
-> 1125             body = body.encode('iso-8859-1')
   1126         self.endheaders(body)
   1127 

UnicodeEncodeError: 'latin-1' codec can't encode characters in position 14-15: ordinal not in range(256)

解决方法：我从“表情”块 U+1F600 - U+1F64F 和符号和象形文字块 U+1F300 - U+1F5FF 中删除所有 unicode 字符this 用下面的代码回答，因为我不需要表情和图片来分析：

text = re.sub(r'[^\u1F600-\u1F64F ]|[^\u1F300-\u1F5FF ]',"",text)

更新 Web 服务的创建者现已修复此问题并更新了文档。您所要做的就是在 Python 3 中发送一个编码字符串：

""Två dagar kvar????????".encode("utf-8")

【问题讨论】：

您可以检查该请求库是否可以使用非 iso-8859-1 的其他编码来执行请求。（我猜它确实如此，现在无法做到这一点将是一个严重的限制）。对于您的解决方法，您需要删除 U+00FF 以上的所有字符，这只会留下拉丁字符集的一小部分。

标签： python-3.x post unicode utf-8 httprequest

【解决方案1】：

尚不清楚 json-tagger.herokuapp.com 期望的内容类型（示例相互矛盾）。您可以尝试将数据作为文本发布：

#!/usr/bin/env python
import requests  # pip install requests

r = requests.post(url,
                  data=text.encode('utf-8'),
                  headers={'Content-type': 'text/plain; charset=utf-8'})
print(r.json())

或者您可以尝试将其发送为application/x-www-form-urlencoded：

#!/usr/bin/env python
import requests  # pip install requests

r = requests.post(url, data=dict(data=text))
print(r.json())

服务器可能同时拒绝、接受两者、接受一个但不接受另一个，或者期待其他格式（例如，application/json），等等。

【讨论】：