如何从 utf-8 中恢复符号

Question

Python 2.7 中的编码很难理解。有人可以向我解释如何获取这些字符串的符号吗？

这是我的 unicode 字符串：

my_str = u'MFADCINEMve000301119 FACTURE EFAD CIN\u2019troD+000000035165 EUR FACTURE EFAD CIN\u2019trop\xe9MA SAS 2019/10198'

我想把它转换成“\u2019”和“\xe9”。

我已经尝试my_str.encode('utf-8') 但这给了我：

'MFADCINEMve000301119 FACTURE EFAD CIN\xe2\x80\x99troD+000000035165 EUR FACTURE EFAD CIN\xe2\x80\x99trop\xc3\xa9MA SAS 2019/10198'

与其他编码符号。我不明白，我只是想将它们替换为 ' 和 é 符号...

更新：

更新 2：

这是我的代码：

day = datetime.now().day
        month = datetime.now().strftime("%b")
        year = datetime.now().strftime("%Y")
        filename = "ventes{0}{1}{2}.csv".format(day, month, year)

        with io.open(filename, 'w', encoding='utf-8') as file_data:
            csvwriter = csv.writer(file_data, delimiter=',', quotechar="", quoting=csv.QUOTE_NONE)

            for line in res:
                csvwriter.writerow([x for x in line])  # Occurs error bellow

        file_data.seek(0)

        out = base64.encodestring(file_data.read())

即出现此错误（不一定明确）：

Traceback (most recent call last):
  File "/usr/lib/python2.7/dist-packages/openerp/http.py", line 546, in _handle_exception
    return super(JsonRequest, self)._handle_exception(exception)
  File "/usr/lib/python2.7/dist-packages/openerp/http.py", line 583, in dispatch
    result = self._call_function(**self.params)
  File "/usr/lib/python2.7/dist-packages/openerp/http.py", line 319, in _call_function
    return checked_call(self.db, *args, **kwargs)
  File "/usr/lib/python2.7/dist-packages/openerp/service/model.py", line 118, in wrapper
    return f(dbname, *args, **kwargs)
  File "/usr/lib/python2.7/dist-packages/openerp/http.py", line 316, in checked_call
    return self.endpoint(*a, **kw)
  File "/usr/lib/python2.7/dist-packages/openerp/http.py", line 812, in __call__
    return self.method(*args, **kw)
  File "/usr/lib/python2.7/dist-packages/openerp/http.py", line 412, in response_wrap
    response = f(*args, **kw)
  File "/usr/lib/python2.7/dist-packages/openerp/addons/web/controllers/main.py", line 953, in call_button
    action = self._call_kw(model, method, args, {})
  File "/usr/lib/python2.7/dist-packages/openerp/addons/web/controllers/main.py", line 941, in _call_kw
    return getattr(request.registry.get(model), method)(request.cr, request.uid, *args, **kwargs)
  File "/usr/lib/python2.7/dist-packages/openerp/api.py", line 268, in wrapper
    return old_api(self, *args, **kwargs)
  File "/usr/lib/python2.7/dist-packages/openerp/api.py", line 399, in old_api
    result = method(recs, *args, **kwargs)
  File "/usr/lib/python2.7/dist-packages/openerp/addons_eggs/adquat_export_CEGID/models/export_cegid.py", line 31, in validate
    move_ids = self._context.get('active_ids', [])
UnicodeEncodeError: 'ascii' codec can't encode character u'\xe9' in position 136: ordinal not in range(128)

这段代码有什么问题？请帮忙！

Answer 1

默认情况下，Python 2 仅将字符串表示形式 (repr()) 显示为 ASCII。 ASCII 范围 (0-127) 之外的任何字符都显示为转义码（\xnn 或 \unnnn）。仅当您 print 字符时，该字符才能正确显示，并且仅当终端编码和字体支持该字符时。

例如：

>>> s = u'\xe9'
>>> s             # This is a representation of the string useful for debugging.
u'\xe9'
>>> len(s)        # It is still only length 1.
1
>>> print(s)      # It displays correctly when printed.
é

默认情况下，我的终端编码不支持所有 Unicode 字符，所以你的另一个例子不支持print。但是，调试表示会：

>>> s = u'\u2019'
>>> s
u'\u2019'
>>> len(s)
1
>>> print(s)
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "D:\dev\Python27\lib\encodings\cp437.py", line 12, in encode
    return codecs.charmap_encode(input,errors,encoding_map)
UnicodeEncodeError: 'charmap' codec can't encode character u'\u2019' in position 0: character maps to <undefined>

如果您将 Unicode 字符串写入文件，则必须对其进行编码。使用您想要的编码打开一个文件并写入 Unicode 字符串。最好使用 UTF-8 作为编码，因为它支持所有 Unicode 字符。使用io.open。它与 Python 3 兼容（您应该尽快切换到它）并支持 encoding 参数。

import io

my_str = u'MFADCINEMve000301119 FACTURE EFAD CIN\u2019troD+000000035165 EUR FACTURE EFAD CIN\u2019trop\xe9MA SAS 2019/10198'
with io.open('out.txt','w',encoding='utf8') as f:
    f.write(my_str)

请注意，您必须在支持 UTF-8 的编辑器中查看文件。例如，在我的终端上使用默认的cp437 编码，它看起来像：

C:\>type out.txt
MFADCINEMve000301119 FACTURE EFAD CIN’troD+000000035165 EUR FACTURE EFAD CIN’tropéMA SAS 2019/10198

但如果我将编码更改为 cp65001 (UTF-8)：

C:\>chcp 65001
Active code page: 65001

C:\>type out.txt
MFADCINEMve000301119 FACTURE EFAD CIN’troD+000000035165 EUR FACTURE EFAD CIN’tropéMA SAS 2019/10198

更多阅读：

• The Absolute Minimum Every Software Developer Absolutely, Positively Must Know About Unicode and Character Sets (No Excuses!)
• Pragmatic Unicode

Answer 2

你只需要print(my_str.encode('utf-8'))

这将为您提供输出：

MFADCINEMve000301119 FACTURE EFAD CIN'troD+000000035165 EUR FACTURE EFAD CIN'tropéMA SAS 2019/10198