从 JSON 写入 CSV 时出现 UnicodeEncodeError

Question

尝试使用 JSON 写入 CSV 时出现以下错误：

Traceback (most recent call last):
File "twitter_search_csv.py", line 25, in <module>
status['retweet_count'],
UnicodeEncodeError: 'ascii' codec can't encode character u'\u2026' in position 139: ordinal not in range(128)

这是我正在使用的代码：

import requests
import urllib2
from requests_oauthlib import OAuth1
import csv

auth = OAuth1('', '', '', '')
url = 'https://api.twitter.com/1.1/search/tweets.json?q=%23OpeningCeremony'

response = requests.get(url, auth=auth)

data = response.json()['statuses']

with open('olympic_search.csv', 'wb') as csvfile:
    f = csv.writer(csvfile)
    for status in data:
        f.writerow([
            status['id'],
            status['text'],
            status['created_at'],
            status['coordinates'],
            status['user']['id_str'],
            status['retweet_count'],
        ])

Answer 1

显式编码字段。否则 Python 会尝试使用 ascii 编码对其进行编码。

>>> print u'\u2026'.encode('ascii')

Traceback (most recent call last):
  File "<pyshell#2>", line 1, in <module>
    print u'\u2026'.encode('ascii')
UnicodeEncodeError: 'ascii' codec can't encode character u'\u2026' in position 0: ordinal not in range(128)
>>> print u'\u2026'.encode('utf-8')
…

---

f.writerow([
    status['id'],
    status['text'].encode('utf-8'), # <----
    status['created_at'],
    status['coordinates'],
    status['user']['id_str'],
    status['retweet_count'],
])