【问题标题】:Need help in inserting Pandas Dataframe into Sql-server DB in python在 python 中将 Pandas Dataframe 插入 Sql-server DB 时需要帮助
【发布时间】:2018-01-21 20:34:02
【问题描述】:

我正在尝试使用 dataframe.to_SQL 将 pandas 数据框 CAPE 插入 SQL Server DB。我已经参考了以下解决方案来插入行。 PyOdbc fails to connect to a sql server instance 。但我在urllib.parse.quote_plus 行中遇到错误。

任何人都可以帮助提供在 Sql-server DB 中插入数据帧的解决方案。

源代码:

   CAPE    # Input dataframe
   connection = pdc.connect('Driver={SQL Server};''Server=GIRSQL.GIRCAPITAL.com;''Database=Tableau;''uid=SQL_User;pwd=Greentableau!')
   connection_string = urllib.parse.quote_plus(connection)
   connection_string = "mssql+pyodbc:///?odbc_connect=%s" % connection_string
   engine = sq.create_engine(connection_string)
   CAPE.to_sql(engine, name='[Tableau].[dbo].[Company_Table]',if_exists='replace')

这是我得到的错误:

    Traceback (most recent call last):
  File "<input>", line 1, in <module>
  File "C:\Users\Abhay\Python36-32\lib\urllib\parse.py", line 803, in quote_plus
string = quote(string, safe + space, encoding, errors)
  File "C:\Users\Abhay\Python36-32\lib\urllib\parse.py", line 787, in quote
 return quote_from_bytes(string, safe)
  File "C:\Users\Abhay\Python36-32\lib\urllib\parse.py", line 812, in quote_from_bytes
  raise TypeError("quote_from_bytes() expected bytes")
  TypeError: quote_from_bytes() expected bytes
  connection = pdc.connect('Driver={SQL Server};''Server=GIRSQL.GIRCAPITAL.com;''Database=Tableau;''uid=SQL_User;pwd=Greentableau!')
   connection_string = ur.quote(connection)
  Traceback (most recent call last):
  File "<input>", line 1, in <module>
  File "C:\Users\Abhay\Python36-32\lib\urllib\parse.py", line 787, in quote
return quote_from_bytes(string, safe)
  File "C:\Users\Abhay\Python36-32\lib\urllib\parse.py", line 812, in 
 quote_from_bytes
  raise TypeError("quote_from_bytes() expected bytes")
  TypeError: quote_from_bytes() expected bytes

示例数据框值:

        Date Company Value     Category BICS_LEVEL_1_SECTOR_NAME BICS_LEVEL_2_INDUSTRY_GROUP_NAME BICS_LEVEL_3_INDUSTRY_NAME BICS_LEVEL_4_SUB_INDUSTRY_NAME BICS_LEVEL_5_SEGMENT_NAME BICS_REVENUE_LEVEL_ASSIGNED BS_TOT_VAL_OF_SHARES_REPURCHASED COUNTRY COUNTRY_OF_LARGEST_REVENUE EQY_SH_OUT GICS_INDUSTRY_GROUP_NAME        GICS_INDUSTRY_NAME GICS_SECTOR_NAME    GICS_SUB_INDUSTRY_NAME      ICB_SECTOR_NAME            INDUSTRY_GROUP INDUSTRY_SECTOR INDUSTRY_SECTOR_NUM        INDUSTRY_SUBGROUP MARKET_SECTOR_DES Real_Earnings Real_Price  CAPE_10  Percentile_10_CAPE
        0 1975-04-30   3M Co     0          EPS                Materials                        Chemicals        Specialty Chemicals           Adhesives & Sealants                       NaN                       10399                          3635.82      US              United States    596.767            Capital Goods  Industrial Conglomerates      Industrials  Industrial Conglomerates  General Industrials  Miscellaneous Manufactur      Industrial               10011  Diversified Manufact Op            Equity             0          0      NaN                 NaN
        1 1975-04-30   3M Co     0  Stock Price                Materials                        Chemicals        Specialty Chemicals           Adhesives & Sealants                       NaN                       10399                          3635.82      US              United States    596.767            Capital Goods  Industrial Conglomerates      Industrials  Industrial Conglomerates  General Industrials  Miscellaneous Manufactur      Industrial               10011  Diversified Manufact Op            Equity             0          0      NaN                 NaN
        2 1975-04-30   3M Co     0    Cash Flow                Materials                        Chemicals        Specialty Chemicals           Adhesives & Sealants                       NaN                       10399                          3635.82      US              United States    596.767            Capital Goods  Industrial Conglomerates      Industrials  Industrial Conglomerates  General Industrials  Miscellaneous Manufactur      Industrial               10011  Diversified Manufact Op            Equity             0          0      NaN                 NaN

我使用的是 SQL Server 版本 13.0.4

版本 2:

我已经更新了我的代码。现在它给sqlalchemy.exc.DBAPIError:

代码:

  import pyodbc
  import sqlalchemy

   CAFE # sample dataframe
   engine = sqlalchemy.create_engine("mssql+pyodbc://SQL_User:Greentableau!@GIRSQL.GIRCAPITAL.com/Tableau?driver=SQL+Server+Native+Client+11.0")
   engine.connect()
   CAPE.to_sql(name='[Tableau].[dbo].[Test_table]',con=engine, if_exists='replace')

错误:

     sqlalchemy.exc.DBAPIError: (pyodbc.Error) ('IM002', '[IM002] [Microsoft][ODBC Driver Manager] Data source name not found and no default driver specified (0) (SQLDriverConnect)')

【问题讨论】:

  • 你的 SQL Server 版本是多少?
  • SQL 服务器版本 13.0.4

标签: python sql-server pandas sql-insert


【解决方案1】:

AFAIK 不需要引用连接字符串。

这是来自SQL Alchemy online docs的示例:

engine = create_engine("mssql+pyodbc://scott:tiger@myhost:port/databasename?driver=SQL+Server+Native+Client+10.0")

即我们不必直接调用/使用 PyODBC - SQL Alchemy 会为我们做这件事......

PSthe driver name will depend on your SQL Server version...

更新:感谢@ArvinthKumar - here is the connection string that finally worked

engine = create_engine('mssql+pyodbc:///?odbc_connect=DRIVER={SQL Server};SERVER=GIRSQL.GIRCAPITAL.com;DATABASE=Tableau;UID=SQ‌​‌​L_User;PWD=sql_password')

【讨论】:

  • 我已将代码更改为关注params = urllib.parse.quote_plus("DRIVER={SQL Server};SERVER=GIRSQL.GIRCAPITAL.com;DATABASE=Tableau;UID=SQ‌​L_User;PWD=Greentabl‌​eau!") engine = sqlalchemy.create_engine("mssql+pyodbc:///?odbc_connect=%s" % params) engine.connect() df.to_sql(name='[Tableau].[dbo].[Test table]',con=engine, index=False, if_exists='append') 现在它可以正常工作了
  • @ArvinthKumar,感谢您的更新!您能否也发布print("mssql+pyodbc:///?odbc_connect=%s" % params) 的输出 - 这可能对将来遇到相同问题的人有所帮助?
  • print("mssql+pyodbc:///?odbc_connect=%s" % params) 语句的输出:Engine(mssql+pyodbc:///?odbc_connect=DRIVER={SQL Server};SERVER=GIRSQL.GIRCAPITAL.com;DATABASE=Tableau;UID=SQL_User;PWD=Greentableau!)
  • @ArvinthKumar,所以如果你尝试conn = create_engine('mssql+pyodbc:///?odbc_connect=DRIVER={SQL Server};SERVER=GIRSQL.GIRCAPITAL.com;DATABASE=Tableau;UID=SQ‌​L_User;PWD=Greentabl‌​eau!') - 它工作正常吗?
  • @ArvinthKumar,感谢您的测试!我会更新我的答案。我希望它可以帮助将来的人......
猜你喜欢
  • 1970-01-01
  • 1970-01-01
  • 2017-09-28
  • 1970-01-01
  • 2012-03-05
  • 1970-01-01
  • 2011-07-10
  • 2016-07-04
  • 2016-04-05
相关资源
最近更新 更多