读取csv文件后插入MySQl数据库？

Question

我有一个这样的 csv 文件：

nohaelprince@uwaterloo.ca, 01-05-2014
nohaelprince@uwaterloo.ca, 01-05-2014
nohaelprince@uwaterloo.ca, 01-05-2014
nohaelprince@gmail.com, 01-05-2014

我需要阅读上面的 csv 文件并提取域名以及按域名和日期划分的电子邮件地址计数。所有这些我都需要插入到 MySQL 数据库中，但是在迭代我得到的列表后，我不知如何插入到 MySQL 数据库中。

查询将是这样的：

INSERT INTO domains(domain_name, cnt, date_of_entry) VALUES (%s, %s, %s);

下面是代码

#!/usr/bin/python
import fileinput
import csv
import os
import sys
import MySQLdb

from collections import defaultdict

lst = defaultdict(list)
d_lst = defaultdict(list)

# ======================== Defined Functions ======================
def get_file_path(filename):
    currentdirpath = os.getcwd()  
    # get current working directory path
    filepath = os.path.join(currentdirpath, filename)
    return filepath
# ===========================================================
def read_CSV(filepath):

   domain_list = []
   domain_date_list = []
   sorted_domain_list_bydate = defaultdict(list)

   with open(filepath, 'rb') as csvfile:
       reader = csv.reader(csvfile)

       for row in reader:
          # insert the 1st & 2nd column of the CSV file into a set called input_list
           email = row[0].strip().lower()
           date  = row[1].strip()

           domain_date_list.append([date, email[ email.find("@") : ]])
           domain_list.append(email[ email.find("@") : ])

   for k, v in domain_date_list: 
         sorted_domain_list_bydate[k].append(v)


   # remove duplicates from domain list
   domain_list = list(set(domain_list))

   return sorted_domain_list_bydate, domain_list
# ===========================================================
def update_DB(lst):

    # open a database connection
    db = MySQLdb.connect(host="localhost", # your host, usually localhost
                         user="root", # your username
                          passwd="abcdef1234", # your password
                          db="test") # name of the data base
    cur = db.cursor() 

    a = []
    for k, v in lst.items():
        # now what should I do here?
        # this is what I am confuse

    db.commit()
    db.close()
# ==========================================================

# ======================= main program =======================================
path = get_file_path('emails.csv') 
[lst, d_lst] = read_CSV(path) # read the input file
update_DB(lst) # insert data into domains table

我对@9​​87654324@ 方法感到困惑。

Answer 1

这里的 read_csv 函数返回 sorteddomainlistbydate 和 domain_list（这是一个列表），由 update_db 函数使用，您可以在其中进行插入。

您的列表只包含域名，而每对键值应包含的内容应包含域名和计数 喜欢

google.com,2

live.com,1

for k, v in lst.items():
     cur.execute("INSERT INTO domains(domain_name, cnt, date_of_entry) VALUES ('" + str(k) + "','" + str(v) + "','" + str(time.strftime("%d/%m/%Y"))+"')")

Answer 2

我不知道您为什么要为一项简单的任务编写如此复杂的程序。让我们从头开始：

1. 您需要先按域、日期正确组织数据，然后计数。

   import csv
   from collections import defuaultdict, Counter
   
   domain_counts = defaultdict(Counter)
   
   with open('somefile.csv') as f:
       reader = csv.reader(f)
       for row in reader:
           domain_counts[row[0].split('@')[1].strip()][row[1]] += 1
   

2. 接下来，需要在数据库中正确插入每一行行：

   db = MySQLdb.connect(...)
   cur = db.cursor()
   
   q = 'INSERT INTO domains(domain_name, cnt, date_of_entry) VALUES(%s, %s, %s)'
   
   for domain, data in domain_counts.iteritems():
       for email_date, email_count in data.iteritems():
             cur.execute(q, (domain, email_count, email_date))
             db.commit()
   

---

由于您的日期未正确插入，请尝试使用此更新后的查询：

q = """INSERT INTO 
          domains(domain_name, cnt, date_of_entry)
          VALUES(%s, %s, STR_TO_DATE(%s, '%d-%m-%Y'))"""