【发布时间】：2020-05-07 13:16:24
【问题描述】：
我目前正在为一个新项目评估 SQLAlchemy。尝试执行包含非 ascii 字符的查询时会引发异常。
用于执行评估的 SQL 语句：
SELECT owner, table_name FROM all_tables WHERE owner LIKE 'äöüßÄÖÜ';
在 SQL*Plus 中执行此语句，SQL Developer 结果——正如预期的那样——在一个空列表中。
为了连接到 Oracle 数据库，正在使用以下代码：
from sqlalchemy import create_engine, MetaData, Table, inspect, select
import pandas as pd
import keyring
dbtype = 'Oracle'
dbenv = 'LOCAL'
dbname = 'MYDB'
dbsys = '%s%s' % (dbtype, dbenv)
dbusr = 'myusr'
dbpwd = keyring.get_password(dbsys, dbusr)
dbhost = 'mydbhost'
dbport = 1521
dbconstr = 'oracle+cx_oracle://%s:%s@%s:%s/%s' % (dbusr, dbpwd, dbhost, dbport, dbname)
评估数据库引擎编码：
dbencs = ['UTF8', 'UTF-8', 'utf8', 'utf-8', 'latin1', 'ascii', None]
for dbenc in dbencs:
if dbenc is None:
engine = create_engine(dbconstr)
else:
engine = create_engine(dbconstr, encoding=dbenc)
con = engine.connect()
try:
df = pd.read_sql_query(u'SELECT owner, table_name FROM all_tables WHERE owner LIKE \'äöüßÄÖÜ\'', con)
print('SUCCESS: sql query with db encoding %s succeeded!' % dbenc)
except Exception as e:
print('ERROR: sql query with db encoding %s failed (%s)' % (dbenc, e))
con.close()
engine.dispose()
无论创建数据库引擎时指定的编码如何，每次尝试执行查询都会引发异常
ERROR: sql query with db encoding UTF8 failed ('ascii' codec can't encode characters in position 60-66: ordinal not in range(128))
ERROR: sql query with db encoding UTF-8 failed ('ascii' codec can't encode characters in position 60-66: ordinal not in range(128))
ERROR: sql query with db encoding utf8 failed ('ascii' codec can't encode characters in position 60-66: ordinal not in range(128))
ERROR: sql query with db encoding utf-8 failed ('ascii' codec can't encode characters in position 60-66: ordinal not in range(128))
ERROR: sql query with db encoding latin1 failed ('ascii' codec can't encode characters in position 60-66: ordinal not in range(128))
ERROR: sql query with db encoding ascii failed ('ascii' codec can't encode characters in position 60-66: ordinal not in range(128))
ERROR: sql query with db encoding None failed ('ascii' codec can't encode characters in position 60-66: ordinal not in range(128))
使用 cx_Oracle 直接连接数据库时（不使用 SQLAlchemy）
import cx_Oracle
import pandas as pd
import keyring
dbtype = 'Oracle'
dbenv = 'LOCAL'
dbname = 'MYDB'
dbsys = '%s%s' % (dbtype, dbenv)
dbusr = 'myusr'
dbpwd = keyring.get_password(dbsys, dbusr)
dbhost = 'mydbhost'
dbport = 1521
dbconstr = '%s:%s/%s' % (dbhost, dbport, dbname)
dbencs = ['UTF8', 'UTF-8', 'utf8', 'utf-8', 'latin1', 'ascii', None]
for dbenc in dbencs:
print('=' * 70)
print('db encoding: %s' % dbenc)
print('-' * 30)
if dbenc is None:
connection = cx_Oracle.connect(dbusr, dbpwd, dbconstr)
else:
connection = cx_Oracle.connect(dbusr, dbpwd, dbconstr, encoding=dbenc)
cursor = connection.cursor()
try:
r = cursor.execute("SELECT owner, table_name FROM all_tables WHERE owner LIKE 'äöüßÄÖÜ'")
recs = list()
for owner, table_name in cursor:
recs.append({'owner': owner, 'table': table_name})
df = pd.DataFrame(recs)
print('SUCCESS: sql query with db encoding %s succeeded!' % dbenc)
except Exception as e:
print('ERROR: sql query with db encoding %s failed (%s)' % (dbenc, e))
cursor.close()
connection.close()
一切都按预期进行。
SUCCESS: sql query with db encoding UTF8 succeeded!
SUCCESS: sql query with db encoding UTF-8 succeeded!
SUCCESS: sql query with db encoding utf8 succeeded!
SUCCESS: sql query with db encoding utf-8 succeeded!
SUCCESS: sql query with db encoding latin1 succeeded!
ERROR: sql query with db encoding ascii failed ('ascii' codec can't encode characters in position 60-66: ordinal not in range(128))
ERROR: sql query with db encoding None failed ('ascii' codec can't encode characters in position 60-66: ordinal not in range(128))
为了让 SQLAlchemy 方法像 cx_Oracle 一样呈现相同的结果，我必须做些什么不同的事情？
我的环境包括
- Ubuntu linux 16.04LTS;
- Python 3.8；
- SQLAlchemy 1.3.16；
- cx_Oracle 7.3.0;
- psycopg2 2.8.5;
- 本地 Oracle 18c Instant 客户端；
- 远程 Oracle 19c 数据库；
- 本地 PostgreSQL 9.5 数据库。
编辑
######################################################################
Traceback (most recent call last):
File "/data/projects/Python/database/sqlalchemy/sqlalchemy_oracle.py", line 45, in <module>
df = pd.read_sql_query(u'SELECT owner, table_name FROM all_tables WHERE owner LIKE \'äöüßÄÖÜ\'', con)
File "/opt/pyenv/versions/3.8.2/lib/python3.8/site-packages/pandas/io/sql.py", line 326, in read_sql_query
return pandas_sql.read_query(
File "/opt/pyenv/versions/3.8.2/lib/python3.8/site-packages/pandas/io/sql.py", line 1218, in read_query
result = self.execute(*args)
File "/opt/pyenv/versions/3.8.2/lib/python3.8/site-packages/pandas/io/sql.py", line 1087, in execute
return self.connectable.execute(*args, **kwargs)
File "/opt/pyenv/versions/3.8.2/lib/python3.8/site-packages/sqlalchemy/engine/base.py", line 976, in execute
return self._execute_text(object_, multiparams, params)
File "/opt/pyenv/versions/3.8.2/lib/python3.8/site-packages/sqlalchemy/engine/base.py", line 1145, in _execute_text
ret = self._execute_context(
File "/opt/pyenv/versions/3.8.2/lib/python3.8/site-packages/sqlalchemy/engine/base.py", line 1287, in _execute_context
self._handle_dbapi_exception(
File "/opt/pyenv/versions/3.8.2/lib/python3.8/site-packages/sqlalchemy/engine/base.py", line 1485, in _handle_dbapi_exception
util.raise_(exc_info[1], with_traceback=exc_info[2])
File "/opt/pyenv/versions/3.8.2/lib/python3.8/site-packages/sqlalchemy/util/compat.py", line 178, in raise_
raise exception
File "/opt/pyenv/versions/3.8.2/lib/python3.8/site-packages/sqlalchemy/engine/base.py", line 1247, in _execute_context
self.dialect.do_execute(
File "/opt/pyenv/versions/3.8.2/lib/python3.8/site-packages/sqlalchemy/engine/default.py", line 590, in do_execute
cursor.execute(statement, parameters)
UnicodeEncodeError: 'ascii' codec can't encode characters in position 60-66: ordinal not in range(128)
######################################################################
EDIT2
引擎对象配置
- convert_unicode = False
- cx_oracle_ver = (7, 3, 0)
- 驱动程序 = cx_oracle
- 编码 = UTF8
- nencoding ... 属性不可用
【问题讨论】：
标签： python sqlalchemy