使用 csv.DictWriter 将字典的 {'key':value} 对写入 .csv 文件并得到“ ValueError: dict contains fields not in fieldnames: '答案

【问题标题】：Using csv.DictWriter to write a dictionary's {'key':value} pairs into a .csv file and get " ValueError: dict contains fields not in fieldnames: '使用 csv.DictWriter 将字典的 {'key':value} 对写入 .csv 文件并得到“ ValueError: dict contains fields not in fieldnames: '
【发布时间】：2020-10-20 06:15:07
【问题描述】：

我正在使用 Bash 生成 python 脚本的输出。 Bash 中的字典如下所示，我觉得很混乱：

error = [('Connection to DB failed', 14), ('Permission denied while closing ticket', 11), ('The ticket was modified while updating', 10), ("Ticket doesn't exist", 8), ('Timeout while retrieving information', 16), ('Tried to add information to closed ticket', 13)]

它一直让我失望，因为 python IDE 中的字典看起来像：

{'a':2, 'b':3, 'as':32, 'bb':13, 'da':32, 'vb':53}

我正在使用下面的代码将 'error' 中的字典键和值存储到一个名为 'error_message.csv' 的文件中。

keys1 = ['Error','Count']
    
with open('error_message.csv', 'w+') as file:
   writer = csv.DictWriter(file, fieldnames=keys1)
   writer.writeheader()
   writer.writerows([error])

我一直收到的错误是：

raise ValueError("dict contains fields not in fieldnames: "
ValueError: dict contains fields not in fieldnames: 'Timeout while retrieving information', 'Permission denied while closing ticket', 'Tried to add information to closed ticket', 'The ticket was modified while updating', 'Connection to DB failed', "Ticket doesn't exist"

在我看来，csv.DictWriter 认为字典的前 2 个条目即

('Connection to DB failed', 14) 
('Permission denied while closing ticket', 11)

是与字段名 ['Error', 'Count'] 关联的字段。这意味着 csv.DictWriter 认为：

'Error' = ('Connection to DB failed', 14) and
'Count' = ('Permission denied while closing ticket', 11)

我已经尝试过这段代码的许多变体（包括只使用一个简单的 csv.writer 来编写每个键值对），其中一个如下：

keys1 = ['Error','Count']

with open('error_message.csv', 'w+') as file:
   writer = csv.DictWriter(file, fieldnames=keys1)
   writer.writeheader()
   for data in error:
      writer.writerow(data)

使用此特定代码，我收到以下错误：

wrong_fields = rowdict.keys() - self.fieldnames
AttributeError: 'str' object has no attribute 'keys

我不确定我在哪里犯了错误。以下代码创建 error_message.csv 文件。

with open('syslog.log') as file:
        for row in file:
                row.strip()
                echeck = re.findall(r'ERROR', row)
                
                if 'ERROR' in echeck:
                        echeck=re.split(r'[ ()]', row)
                        
                        user=echeck[-2]
                        user.strip()
                        
                        if user not in per_user:
                           per_user[user]=1
                        per_user[user]+=1
                        
                        echeck=re.split(r'[ERROR()]',row)
                        errors=echeck[-3].strip()
                        
                        if errors not in error:
                           error[errors]=1
                        error[errors]+=1

“syslog.log”文件包含如下所示的系统日志：

Jan 31 00:09:39 ubuntu.local ticky: INFO Created ticket [#4217] (mdouglas)
Jan 31 00:16:25 ubuntu.local ticky: INFO Closed ticket [#1754] (noel)
Jan 31 00:21:30 ubuntu.local ticky: ERROR The ticket was modified while updating (breee)
Jan 31 00:44:34 ubuntu.local ticky: ERROR Permission denied while closing ticket (ac)
Jan 31 01:00:50 ubuntu.local ticky: INFO Commented on ticket [#4709] (blossom)
Jan 31 01:29:16 ubuntu.local ticky: INFO Commented on ticket [#6518] (rr.robinson)
Jan 31 01:33:12 ubuntu.local ticky: ERROR Tried to add information to closed ticket (mcintosh)
Jan 31 01:43:10 ubuntu.local ticky: ERROR Tried to add information to closed ticket (jackowens)
Jan 31 01:49:29 ubuntu.local ticky: ERROR Tried to add information to closed ticket (mdouglas)
Jan 31 02:30:04 ubuntu.local ticky: ERROR Timeout while retrieving information (oren)
Jan 31 02:55:31 ubuntu.local ticky: ERROR Ticket doesn't exist (xlg)
Jan 31 03:05:35 ubuntu.local ticky: ERROR Timeout while retrieving information (ahmed.miller)
Jan 31 03:08:55 ubuntu.local ticky: ERROR Ticket doesn't exist (blossom)
Jan 31 03:39:27 ubuntu.local ticky: ERROR The ticket was modified while updating (bpacheco)
Jan 31 03:47:24 ubuntu.local ticky: ERROR Ticket doesn't exist (enim.non)
Jan 31 04:30:04 ubuntu.local ticky: ERROR Permission denied while closing ticket (rr.robinson)
Jan 31 04:31:49 ubuntu.local ticky: ERROR Tried to add information to closed ticket (oren)
Jan 31 04:32:49 ubuntu.local ticky: ERROR Timeout while retrieving information (mcintosh)
Jan 31 04:44:23 ubuntu.local ticky: ERROR Timeout while retrieving information (ahmed.miller)
Jan 31 04:44:46 ubuntu.local ticky: ERROR Connection to DB failed (jackowens)
Jan 31 04:49:28 ubuntu.local ticky: ERROR Permission denied while closing ticket (flavia)
Jan 31 05:12:39 ubuntu.local ticky: ERROR Tried to add information to closed ticket (oren)
Jan 31 05:18:45 ubuntu.local ticky: ERROR Tried to add information to closed ticket (sri)
Jan 31 05:23:14 ubuntu.local ticky: INFO Commented on ticket [#1097] (breee)
Jan 31 05:35:00 ubuntu.local ticky: ERROR Connection to DB failed (nonummy)
Jan 31 05:45:30 ubuntu.local ticky: INFO Created ticket [#7115] (noel)
Jan 31 05:51:30 ubuntu.local ticky: ERROR The ticket was modified while updating (flavia)
Jan 31 05:57:46 ubuntu.local ticky: INFO Commented on ticket [#2253] (nonummy)
Jan 31 06:12:02 ubuntu.local ticky: ERROR Connection to DB failed (oren)
Jan 31 06:26:38 ubuntu.local ticky: ERROR Timeout while retrieving information (xlg)
Jan 31 06:32:26 ubuntu.local ticky: INFO Created ticket [#7298] (ahmed.miller)
Jan 31 06:36:25 ubuntu.local ticky: ERROR Timeout while retrieving information (flavia)
Jan 31 06:57:00 ubuntu.local ticky: ERROR Connection to DB failed (jackowens)
Jan 31 06:59:57 ubuntu.local ticky: INFO Commented on ticket [#7255] (oren)
Jan 31 07:59:56 ubuntu.local ticky: ERROR Ticket doesn't exist (flavia)
Jan 31 08:01:40 ubuntu.local ticky: ERROR Tried to add information to closed ticket (jackowens)
Jan 31 08:03:19 ubuntu.local ticky: INFO Closed ticket [#1712] (britanni)
Jan 31 08:22:37 ubuntu.local ticky: INFO Created ticket [#2860] (mcintosh)
Jan 31 08:28:07 ubuntu.local ticky: ERROR Timeout while retrieving information (montanap)
Jan 31 08:49:15 ubuntu.local ticky: ERROR Permission denied while closing ticket (britanni)
Jan 31 08:50:50 ubuntu.local ticky: ERROR Permission denied while closing ticket (montanap)
Jan 31 09:04:27 ubuntu.local ticky: ERROR Tried to add information to closed ticket (noel)
Jan 31 09:15:41 ubuntu.local ticky: ERROR Timeout while retrieving information (oren)
Jan 31 09:18:47 ubuntu.local ticky: INFO Commented on ticket [#8385] (mdouglas)
Jan 31 09:28:18 ubuntu.local ticky: INFO Closed ticket [#2452] (jackowens)
Jan 31 09:41:16 ubuntu.local ticky: ERROR Connection to DB failed (ac)
Jan 31 10:11:35 ubuntu.local ticky: ERROR Timeout while retrieving information (blossom)
Jan 31 10:21:36 ubuntu.local ticky: ERROR Permission denied while closing ticket (montanap)
Jan 31 11:04:02 ubuntu.local ticky: ERROR Tried to add information to closed ticket (breee)
Jan 31 11:19:37 ubuntu.local ticky: ERROR Connection to DB failed (sri)
Jan 31 11:22:06 ubuntu.local ticky: ERROR Timeout while retrieving information (montanap)
Jan 31 11:31:34 ubuntu.local ticky: ERROR Permission denied while closing ticket (ahmed.miller)
Jan 31 11:40:25 ubuntu.local ticky: ERROR Connection to DB failed (mai.hendrix)
Jan 31 11:47:07 ubuntu.local ticky: INFO Commented on ticket [#4562] (ac)
Jan 31 11:58:33 ubuntu.local ticky: ERROR Tried to add information to closed ticket (ahmed.miller)
Jan 31 12:00:17 ubuntu.local ticky: INFO Created ticket [#7897] (kirknixon)
Jan 31 12:02:49 ubuntu.local ticky: ERROR Permission denied while closing ticket (mai.hendrix)
Jan 31 12:20:23 ubuntu.local ticky: ERROR Connection to DB failed (kirknixon)
Jan 31 12:20:40 ubuntu.local ticky: ERROR Ticket doesn't exist (flavia)
Jan 31 12:24:32 ubuntu.local ticky: INFO Created ticket [#5784] (sri)
Jan 31 12:50:10 ubuntu.local ticky: ERROR Permission denied while closing ticket (blossom)
Jan 31 12:58:16 ubuntu.local ticky: ERROR Tried to add information to closed ticket (nonummy)
Jan 31 13:08:10 ubuntu.local ticky: INFO Closed ticket [#8685] (rr.robinson)
Jan 31 13:48:45 ubuntu.local ticky: ERROR The ticket was modified while updating (breee)
Jan 31 14:13:00 ubuntu.local ticky: INFO Commented on ticket [#4225] (noel)
Jan 31 14:38:50 ubuntu.local ticky: ERROR The ticket was modified while updating (enim.non)
Jan 31 14:41:18 ubuntu.local ticky: ERROR Timeout while retrieving information (xlg)
Jan 31 14:45:55 ubuntu.local ticky: INFO Closed ticket [#7948] (noel)
Jan 31 14:50:41 ubuntu.local ticky: INFO Commented on ticket [#8628] (noel)
Jan 31 14:56:35 ubuntu.local ticky: ERROR Tried to add information to closed ticket (noel)
Jan 31 15:27:53 ubuntu.local ticky: ERROR Ticket doesn't exist (blossom)
Jan 31 15:28:15 ubuntu.local ticky: ERROR Permission denied while closing ticket (enim.non)
Jan 31 15:44:25 ubuntu.local ticky: INFO Closed ticket [#7333] (enim.non)
Jan 31 16:17:20 ubuntu.local ticky: INFO Commented on ticket [#1653] (noel)
Jan 31 16:19:40 ubuntu.local ticky: ERROR The ticket was modified while updating (mdouglas)
Jan 31 16:24:31 ubuntu.local ticky: INFO Created ticket [#5455] (ac)
Jan 31 16:35:46 ubuntu.local ticky: ERROR Timeout while retrieving information (oren)
Jan 31 16:53:54 ubuntu.local ticky: INFO Commented on ticket [#3813] (mcintosh)
Jan 31 16:54:18 ubuntu.local ticky: ERROR Connection to DB failed (bpacheco)
Jan 31 17:15:47 ubuntu.local ticky: ERROR The ticket was modified while updating (mcintosh)
Jan 31 17:29:11 ubuntu.local ticky: ERROR Connection to DB failed (oren)
Jan 31 17:51:52 ubuntu.local ticky: INFO Closed ticket [#8604] (mcintosh)
Jan 31 18:09:17 ubuntu.local ticky: ERROR The ticket was modified while updating (noel)
Jan 31 18:43:01 ubuntu.local ticky: ERROR Ticket doesn't exist (nonummy)
Jan 31 19:00:23 ubuntu.local ticky: ERROR Timeout while retrieving information (blossom)
Jan 31 19:20:22 ubuntu.local ticky: ERROR Timeout while retrieving information (mai.hendrix)
Jan 31 19:59:06 ubuntu.local ticky: INFO Created ticket [#6361] (enim.non)
Jan 31 20:02:41 ubuntu.local ticky: ERROR Timeout while retrieving information (xlg)
Jan 31 20:21:55 ubuntu.local ticky: INFO Commented on ticket [#7159] (ahmed.miller)
Jan 31 20:28:26 ubuntu.local ticky: ERROR Connection to DB failed (breee)
Jan 31 20:35:17 ubuntu.local ticky: INFO Created ticket [#7737] (nonummy)
Jan 31 20:48:02 ubuntu.local ticky: ERROR Connection to DB failed (mdouglas)
Jan 31 20:56:58 ubuntu.local ticky: INFO Closed ticket [#4372] (oren)
Jan 31 21:00:23 ubuntu.local ticky: INFO Commented on ticket [#2389] (sri)
Jan 31 21:02:06 ubuntu.local ticky: ERROR Connection to DB failed (breee)
Jan 31 21:20:33 ubuntu.local ticky: INFO Closed ticket [#3297] (kirknixon)
Jan 31 21:29:24 ubuntu.local ticky: ERROR The ticket was modified while updating (blossom)
Jan 31 22:58:55 ubuntu.local ticky: INFO Created ticket [#2461] (jackowens)
Jan 31 23:25:18 ubuntu.local ticky: INFO Closed ticket [#9876] (blossom)
Jan 31 23:35:40 ubuntu.local ticky: INFO Created ticket [#5896] (mcintosh)

我正在使用正则表达式在 'syslog.log' 文件中包含的每一行中查找单词 'ERROR'。从所有这些行中，我继续提取错误消息以及生成错误的用户的用户名。

error={} 字典存储所有不同的错误名称，而 per_user={} 字典存储生成 ERROR 的所有不同用户名强>。

【问题讨论】：

似乎error_message.csv 将“与数据库的连接失败”等作为列标题。你能显示文件的内容，以及它是如何创建的吗？
刚刚在那里进行了更新。谢谢@snakecharmerb

标签： python bash csv dictionary

【解决方案1】：

从 Python DictWriter 文档中找到 here

创建一个像普通写入器一样操作但将字典映射到输出行的对象。 fieldnames 参数是一个键序列，用于标识传递给writerow() 方法的字典中的值写入文件 f 的顺序。可选的restval 参数指定在字典缺少字段名中的键时要写入的值。如果传递给 writerow() 方法的字典包含字段名中未找到的键，则可选的 extrasaction 参数指示要采取的操作。如果设置为“raise”，默认值，则会引发 ValueError。如果设置为“忽略”，则字典中的额外值将被忽略。任何其他可选或关键字参数都将传递给底层编写器实例。

因此，fieldnames 参数是将包含在将传递给 writerow 的字典中的键。因为您传入的字典包含除Error 和Count 之外的额外键，因此引发了错误。如果你有动态列，我建议你使用像 pandas 这样的库。

【讨论】：

谢谢奥马尔。我以为字典只有两个键，即“错误”和“计数”。不是这样吗，因为所有“错误”字典成员都包含具有一对一关系的键值对，即一个键只有一个值。

【解决方案2】：

就您的 csv 而言，error 字典是 value:value 而不是 key:value。您的代码只需稍作修改即可使用此结构：使用 csv.writer 而不是 csv.DictWriter 并将字典条目视为要添加到 csv 的行。

with open('error_message.csv', 'w+') as file:
    writer = csv.writer(file)
    writer.writerow(keys1)
    for errmsg, count in error.items():
        writer.writerow([errmsg, count])

一个小技巧：您可以使用defaultdict 跳过检查计数器字典中是否存在键：

import collections

error = collections.defaultdict(int)
per_user = collections.defaultdict(int)

with open('syslog.log') as file:
    for row in file:
        row.strip()
        echeck = re.findall(r'ERROR', row)

        if 'ERROR' in echeck:
            echeck = re.split(r'[ ()]', row)

            user = echeck[-2]
            user.strip()

            per_user[user] += 1

            echeck = re.split(r'[ERROR()]', row)
            errors = echeck[-3].strip()

            error[errors] += 1

这也暴露了原始逻辑中的一个错误 - 对于一个新条目，它将设置 error[errors] = 1 然后立即递增它，因此第一个条目将被计算两次。

【讨论】：