我有一本像下面这样的字典：
{"A": ["a", "b", "c"],
"B": ["a", "d", "f"],
"C": ["i", "i", "j"]}
我想把它改成如下：
{"A": ["b", "c"],
"B": ["d", "f"],
"C": ["j"]}
也就是说，所有重复的值都被删除，无论它们出现在同一个键还是不同的键中。如何高效实现？
【问题讨论】：
collections.Counter计算值，然后用它来过滤每个列表。
标签： python python-3.x dictionary duplicates
此代码将删除所有重复项：
from collections import Counter
def remove_all_dupes(d):
c = Counter()
for v in d.values():
c.update(v)
for k,v in d.items():
d[k] = [item for item in v if c[item] == 1]
return d
d = {"A": ["a", "b", "c"],
"B": ["a", "d", "f"],
"C": ["i", "i", "j"]}
print(d)
remove_all_dupes(d)
print(d)
按要求输出。
我希望这是相当有效的，即O(n)，因为它只循环遍历所有值两次，消除重复的查找应该是O(1)。
【讨论】：
试试这个：
from itertools import chain
d = {"A": ["a", "b", "c"],
"B": ["a", "d", "f"],
"C": ["i", "i", "j"]}
new_dic = {}
values = list(chain(*d.values()))
for key, value in d.items():
new_dic[key] = [x for x in value if values.count(x) == 1]
print(new_dic)
使用字典理解：
new_dic = {key: [x for x in value if list(chain(*d.values())).count(x)==1] for key, value in d.items()}
输出：
{"A": ["b", "c"], "B": ["d", "f"], "C": ["j"]}
【讨论】：
.count(x) 必须为每个x 扫描整个values。
此代码不使用任何导入。
data = {"A": ["a", "b", "c"], "B": ["a", "d", "f"], "C": ["i", "i", "j"]}
# vals = []
# for ky in data.keys():
# vals = vals + data[ky]
vals = sum(data.values(), [])
dups = set([val for val in vals if vals.count(val) > 1])
data_deduped = {
ky: [val for val in data[ky] if not val in dups] for ky in data.keys()
}
print(data_deduped)
Sample Output
{'A': ['b', 'c'], 'B': ['d', 'f'], 'C': ['j']}
【讨论】：
您可以使用pip install more-itertools 和flatten 快速创建所有值的列表，并使用collections 中的Counter 对它们进行计数。
from more_itertools import flatten
from collections import Counter
def remove_duplicates(d):
all_values = list(flatten(d.values()))
count = Counter(all_values)
filtered_dict = {key: [v for v in value if count[v] == 1] for key, value in d.items()}
return filtered_dict
【讨论】：
from collections import Counter
d_input = {"A": ["a", "b", "c"], "B": ["a", "d", "f"], "C": ["i", "i", "j"]}
d_correct_answer = {"A": ["b", "c"], "B": ["d", "f"], "C": ["j"]}
# first make a giant list of all lists in dictionary
giant_list = []
for key in d_input:
partial_list = d_input[key]
# build a big list of all items
giant_list.extend(partial_list)
# use counter to find items
counted = Counter(giant_list)
# now build remove list
remove_list = []
for key in counted:
if counted[key] > 1:
remove_list.append(key)
# now loop and remove by only adding proper results to new dict
result = {}
for key in d_input:
partial_list = d_input[key]
new_partial_list = []
for item in partial_list:
if item not in remove_list:
new_partial_list.append(item)
result[key] = new_partial_list
print(f'it is {result == d_correct_answer} that this code works')
【讨论】：