数据框未插入数据库但抛出 DPI 未连接错误

Question

我需要在 oracle sql 中插入一个数据帧，我能找到的最快方法是 sqlalchemy，但这似乎对我不利。

我的代码：

import pandas as pd 
from sqlalchemy.engine import create_engine
from sqlalchemy import types
import cx_Oracle

DIALECT = 'oracle'
SQL_DRIVER = 'cx_oracle'
USERNAME = 'NCSWEB' #enter your username
PASSWORD = '########' #enter your password
HOST = '192.168.213.151' #enter the oracle db host url
PORT = 1515 # enter the oracle port number
SERVICE = 'devnc12c' # enter the oracle db service name
ENGINE_PATH_WIN_AUTH = DIALECT + '+' + SQL_DRIVER + '://' + USERNAME + ':' + PASSWORD +'@' + HOST + ':' + str(PORT) + '/?service_name=' + SERVICE

engine = create_engine(ENGINE_PATH_WIN_AUTH)
print(engine)
#read data from csv
fileName='C:\\Users\\ncdex1124\Desktop\\New folder\\file4.csv'
df = pd.read_csv(fileName)
print(df)
dtyp = {c:types.VARCHAR(df[c].str.len().max())
    for c in df.columns[df.dtypes == 'object'].tolist()}
try:
    df.to_sql('UCIDBA.FUT_CLIENT_LIST_COPY', engine.connect(), if_exists='append', index=False, chunksize=10000,dtype=dtyp)
except Exception as e:
    print(e)
print("Record inserted successfully")

输出：

        Traceback (most recent call last):
  File "C:\Users\ncdex1124\AppData\Local\Programs\Python\Python39\lib\site-packages\sqlalchemy\engine\base.py", line 1751, in _execute_context
    self.dialect.do_executemany(
  File "C:\Users\ncdex1124\AppData\Local\Programs\Python\Python39\lib\site-packages\sqlalchemy\dialects\oracle\cx_oracle.py", line 1347, in do_executemany        
    cursor.executemany(statement, parameters)
cx_Oracle.DatabaseError: DPI-1080: connection was closed by ORA-3135

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "C:\Users\ncdex1124\AppData\Local\Programs\Python\Python39\lib\site-packages\sqlalchemy\engine\base.py", line 1866, in _safe_close_cursor
    cursor.close()
cx_Oracle.DatabaseError: DPI-1010: not connected
(cx_Oracle.DatabaseError) DPI-1010: not connected
(Background on this error at: https://sqlalche.me/e/14/4xp6)

我不明白在这种情况下出了什么问题。

Answer 1

默认情况下，cx_oracle 在没有显式提交的情况下回滚任何事务。

自动提交模式

Connection 对象有一个属性叫做 autocommit 允许您自动提交事务。经过 默认情况下，其值设置为 False。指示 cx_Oracle 提交 交易自动，你设置的价值 Connection.autocommit 为 True 如下：

connection.autocommit = True

当您使用 Pandas 和来自 SQL Alchemy 的 create_engine 时，您需要

connection = engine.connect()
connection = connection.execution_options(
    isolation_level="AUTOCOMMIT"
)

关于您的代码，我会尝试这个，同时考虑到您使用的是最新版本的 Pandas 和 Sql Alchemy。它应该可以工作。

import pandas as pd 
from sqlalchemy.engine import create_engine
from sqlalchemy import types
import cx_Oracle
DIALECT = 'oracle'
SQL_DRIVER = 'cx_oracle'
USERNAME = 'NCSWEB' #enter your username
PASSWORD = '########' #enter your password
HOST = '192.168.213.151' #enter the oracle db host url
PORT = 1515 # enter the oracle port number
SERVICE = 'devnc12c' # enter the oracle db service name
ENGINE_PATH_WIN_AUTH = DIALECT + '+' + SQL_DRIVER + '://' + USERNAME + ':' + PASSWORD +'@' + HOST + ':' + str(PORT) + '/?service_name=' + SERVICE

engine = create_engine(ENGINE_PATH_WIN_AUTH,max_identifier_length=128)
print(engine)
#read data from csv
fileName='C:\\Users\\ncdex1124\Desktop\\New folder\\file4.csv'
df = pd.read_csv(fileName)
print(df)
dtyp = {c:types.VARCHAR(df[c].str.len().max())
    for c in df.columns[df.dtypes == 'object'].tolist()}
try:
    engine = engine.connect().execution_options(autocommit=True) 
    df.to_sql('UCIDBA.FUT_CLIENT_LIST_COPY', con=engine, if_exists='append', index=False, chunksize=10000,dtype=dtyp)
except Exception as e:
    print(e)
print("Record inserted successfully")

一个示例如何在我自己的环境中使用自动提交进行连接

C:\python>python
Python 3.8.1 (tags/v3.8.1:1b293b6, Dec 18 2019, 23:11:46) [MSC v.1916 64 bit (AMD64)] on win32
Type "help", "copyright", "credits" or "license" for more information.
>>> import pandas as pd
>>> from sqlalchemy.engine import create_engine
>>> from sqlalchemy import types
>>> import cx_Oracle
>>> DIALECT = 'oracle'
>>> SQL_DRIVER = 'cx_oracle'
>>> USERNAME = 'PYTHON' #enter your username
>>> PASSWORD = '#######' #enter your password
>>> HOST = 'ODCGRC1R.SCGER.DEV.CORP' #enter the oracle db host url
>>> PORT = 60995 # enter the oracle port number
>>> SERVICE = 'odcgrc1r' # enter the oracle db service name
>>> ENGINE_PATH_WIN_AUTH = DIALECT + '+' + SQL_DRIVER + '://' + USERNAME + ':' + PASSWORD +'@' + HOST + ':' + str(PORT) + '/?service_name=' + SERVICE
>>> engine = create_engine(ENGINE_PATH_WIN_AUTH,max_identifier_length=128)
>>> print(engine)
Engine(oracle+cx_oracle://PYTHON:***@ODCGRC1R.SCGER.DEV.CORP:60995/?service_name=odcgrc1r)
>>> #read data from csv
>>> engine = engine.connect().execution_options(autocommit=True)
>>>