将 MS SQL Server 视图读取到 Pandas 数据框

Question

我正在尝试将 MS SQL Server 视图读取到 pandas 数据框。一开始我以为是表，所以写了如下代码（表/视图、服务器、数据库、ID和密码都改了，去掉了专有信息）：

import sqlalchemy as sa
import urllib
from datetime import date

today = date.today().isformat()

params = urllib.parse.quote_plus('DRIVER={ODBC Driver 17 for SQL Server};'
                                 + 'SERVER=mssql_server;DATABASE=database;UID='
                                 + 'user_id;PWD=password')
engine = sa.create_engine(f'mssql+pyodbc:///?odbc_connect={params}')

with engine.connect() as conn, conn.begin():
    # Read data and store in pandas dataframe.
    df1 = pd.read_sql_table('dbo.view1', conn)
    df2 = pd.read_sql_table('dbo.view2', conn)
    df3 = pd.read_sql_query('SELECT * FROM dbo.view3 WHERE date=' + today, conn)

但它导致了以下错误：

Traceback (most recent call last):

  File "C:\ProgramData\Anaconda3\lib\site-packages\pandas\io\sql.py", line 243, in read_sql_table
    meta.reflect(only=[table_name], views=True)

  File "C:\ProgramData\Anaconda3\lib\site-packages\sqlalchemy\sql\schema.py", line 4261, in reflect
    "in %r%s: (%s)" % (bind.engine, s, ", ".join(missing))

InvalidRequestError: Could not reflect: requested table(s) not available in Engine(mssql+pyodbc:///?odbc_connect=DRIVER={ODBC Driver 17 for SQL Server};SERVER=mssql_server;DATABASE=database;UID=user_id;PWD=password): (dbo.view1)

在处理上述异常的过程中，又发生了一个异常：

Traceback (most recent call last):

  File "<ipython-input-89-bbb0170e1399>", line 1, in <module>
    df1 = pd.read_sql_table('dbo.view1', engine)

  File "C:\ProgramData\Anaconda3\lib\site-packages\pandas\io\sql.py", line 245, in read_sql_table
    raise ValueError("Table {name} not found".format(name=table_name))

ValueError: Table dbo.view1 not found

那时我发现我正在尝试加载视图而不是表格，所以我将代码更改为：

import sqlalchemy as sa
import urllib
from datetime import date

today = date.today().isformat()

params = urllib.parse.quote_plus('DRIVER={ODBC Driver 17 for SQL Server};'
                                 + 'SERVER=mssql_server;DATABASE=database;UID='
                                 + 'user_id;PWD=password')
engine = sa.create_engine(f'mssql+pyodbc:///?odbc_connect={params}')

with engine.connect() as conn, conn.begin():
    # Read data and store in pandas dataframe.
    df1 = pd.read_sql('dbo.view1', conn)
    df2 = pd.read_sql('dbo.view2', conn)
    df3 = pd.read_sql('SELECT * FROM dbo.view3 WHERE date=' + today, conn)

导致以下错误：

Traceback (most recent call last):

  File "C:\ProgramData\Anaconda3\lib\site-packages\sqlalchemy\engine\base.py", line 1246, in _execute_context
    cursor, statement, parameters, context

  File "C:\ProgramData\Anaconda3\lib\site-packages\sqlalchemy\engine\default.py", line 581, in do_execute
    cursor.execute(statement, parameters)

ProgrammingError: ('42000', "[42000] [Microsoft][ODBC Driver 17 for SQL Server][SQL Server]The request for procedure 'view1' failed because 'view1' is a view object. (2809) (SQLExecDirectW)")


The above exception was the direct cause of the following exception:

Traceback (most recent call last):

  File "<ipython-input-76-6b8d4a0d911d>", line 1, in <module>
    df1 = pd.read_sql('dbo.view1', engine)

  File "C:\ProgramData\Anaconda3\lib\site-packages\pandas\io\sql.py", line 436, in read_sql
    chunksize=chunksize,

  File "C:\ProgramData\Anaconda3\lib\site-packages\pandas\io\sql.py", line 1218, in read_query
    result = self.execute(*args)

  File "C:\ProgramData\Anaconda3\lib\site-packages\pandas\io\sql.py", line 1087, in execute
    return self.connectable.execute(*args, **kwargs)

  File "C:\ProgramData\Anaconda3\lib\site-packages\sqlalchemy\engine\base.py", line 2182, in execute
    return connection.execute(statement, *multiparams, **params)

  File "C:\ProgramData\Anaconda3\lib\site-packages\sqlalchemy\engine\base.py", line 976, in execute
    return self._execute_text(object_, multiparams, params)

  File "C:\ProgramData\Anaconda3\lib\site-packages\sqlalchemy\engine\base.py", line 1149, in _execute_text
    parameters,

  File "C:\ProgramData\Anaconda3\lib\site-packages\sqlalchemy\engine\base.py", line 1250, in _execute_context
    e, statement, parameters, cursor, context

  File "C:\ProgramData\Anaconda3\lib\site-packages\sqlalchemy\engine\base.py", line 1476, in _handle_dbapi_exception
    util.raise_from_cause(sqlalchemy_exception, exc_info)

  File "C:\ProgramData\Anaconda3\lib\site-packages\sqlalchemy\util\compat.py", line 398, in raise_from_cause
    reraise(type(exception), exception, tb=exc_tb, cause=cause)

  File "C:\ProgramData\Anaconda3\lib\site-packages\sqlalchemy\util\compat.py", line 152, in reraise
    raise value.with_traceback(tb)

  File "C:\ProgramData\Anaconda3\lib\site-packages\sqlalchemy\engine\base.py", line 1246, in _execute_context
    cursor, statement, parameters, context

  File "C:\ProgramData\Anaconda3\lib\site-packages\sqlalchemy\engine\default.py", line 581, in do_execute
    cursor.execute(statement, parameters)

ProgrammingError: (pyodbc.ProgrammingError) ('42000', "[42000] [Microsoft][ODBC Driver 17 for SQL Server][SQL Server]The request for procedure 'view1' failed because 'view1' is a view object. (2809) (SQLExecDirectW)")
[SQL: dbo.view1]

我尝试过寻找答案，但要么没有找到，要么找错了地方。可以读取熊猫数据框的视图吗？如果是这样，我做错了什么？提前致谢！

Answer 1

这样做怎么样？

import pypyodbc 
cnxn = pypyodbc.connect("Driver={SQL Server Native Client 11.0};"
                        "Server=LAPTOP-CEDUMII6;"
                        "Database=TestDB;"
                        "Trusted_Connection=yes;")


cursor = cnxn.cursor()
dfObj = pd.DataFrame(cursor.execute('SELECT * FROM myTable1'))