将二进制数据插入longblob列时出现pymysql编码错误答案

【问题标题】：pymysql encoding error while inserting binary data into longblob column将二进制数据插入longblob列时出现pymysql编码错误
【发布时间】：2015-07-05 06:45:01
【问题描述】：

我正在尝试将二进制文件的内容插入到 longblob 列中：

Python 代码：

conn = pymysql.connect(...)
cursor = conn.cursor()
with open('test.bz2', 'rb') as fp:
    data = fp.read()
cursor.execute('insert into test_t (test) values (%s)', [data])

错误堆栈跟踪：

Traceback (most recent call last):
  File "./doit2", line 9, in <module>
    cursor.execute('insert into test_t (test) values (%s)', [data])
  File "/u02/srm_tp/local/lib/python3.4/site-packages/pymysql/cursors.py", line 127, in execute
    result = self._query(query)
  File "/u02/srm_tp/local/lib/python3.4/site-packages/pymysql/cursors.py",     line 275, in _query
    conn.query(q)
  File "/u02/srm_tp/local/lib/python3.4/site-packages/pymysql/connections.py", line 763, in query
    sql = sql.encode(self.encoding)
UnicodeEncodeError: 'latin-1' codec can't encode character '\udcae' in position 45: ordinal not in range(256)

创建表脚本：

mysql> show create table test_t;
+--------+--------------------------------------------------------------------------+
| Table  | Create Table                                                             |
+--------+--------------------------------------------------------------------------+
| test_t | CREATE TABLE `test_t` (
  `test` longblob
) ENGINE=InnoDB DEFAULT CHARSET=utf8 COLLATE=utf8_unicode_ci |
+--------+--------------------------------------------------------------------------+

默认编码：

=->python3 -c 'import sys; print(sys.getdefaultencoding())'
utf-8

添加"charset='utf8', use_unicode=True" 连接调用，将错误改为：

Traceback (most recent call last):
  File "./doit2", line 13, in <module>
    cursor.execute('insert into test_t (test) values (%s)', [data])
  File "/u02/srm_tp/local/lib/python3.4/site-packages/pymysql/cursors.py", line 127, in execute
    result = self._query(query)
  File "/u02/srm_tp/local/lib/python3.4/site-packages/pymysql/cursors.py", line 275, in _query
    conn.query(q)
  File "/u02/srm_tp/local/lib/python3.4/site-packages/pymysql/connections.py", line 763, in query
    sql = sql.encode(self.encoding)
UnicodeEncodeError: 'utf-8' codec can't encode character '\udcae' in position 45: surrogates not allowed

【问题讨论】：

标签： python mysql pymysql

【解决方案1】：

This 应该可以解决问题：

conn = pymysql.connect(...)
conn.set_character_set('utf8')
cursor = conn.cursor()
cursor.execute('SET NAMES utf8;') 
cursor.execute('SET CHARACTER SET utf8;')
cursor.execute('SET character_set_connection=utf8;')
with open('test.bz2', 'rb') as fp:
    data = fp.read()
cursor.execute('insert into test_t (test) values (%s)', [data])

【讨论】：

pymysql 没有 set_character_set 功能。我尝试在连接调用中添加“charset='utf8', use_unicode=True”，错误更改为：UnicodeEncodeError: 'utf-8' codec can't encode character '\udcae' in position 45: surrogates not allowed
尝试连接charset="utf8"，例如：db = pymysql.connect(host="localhost", user = "root", passwd = "", db = "testdb", use_unicode=True, charset="utf8")
是的，我做到了。抱歉，在我输入完回复之前按 Enter。现在我得到： UnicodeEncodeError: 'utf-8' codec can't encode character '\udcae' in position 45: surrogates not allowed
尝试运行yourstring.encode('utf-8')，然后再尝试将其放入数据库。
这是奇怪的部分——我不是在这里处理字符串。该文件以二进制形式打开，当我从中读取时 - “数据”是字节。

【解决方案2】：

看起来这是一个 pymysql 错误。我从 0.6.4 升级到 0.6.6（截至目前的最新版本），问题不再存在。

【讨论】：