用公共键合并python字典

Question

假设我有以下字典：

{name: "john", place: "nyc", owns: "gold", quantity: 30}
{name: "john", place: "nyc", owns: "silver", quantity: 20}
{name: "jane", place: "nyc", owns: "platinum", quantity: 5}
{name: "john", place: "chicago", owns: "brass", quantity: 60}
{name: "john", place: "chicago", owns: "silver", quantity: 40}

我有数百本这样的小词典。我必须将它们与公共键的子集合并，在这个例子中（名称，地点）并创建一个新字典。最终，输出应如下所示：

{name: "john", place: "nyc", gold: 30, silver: 20}
{name: "jane", place: "nyc", platinum: 5}
{name: "john", place: "chicago", brass: 60, silver: 40}

有没有有效的方法来做到这一点？我能想到的只是蛮力，我将跟踪每个可能的名称-地点组合，存储在某个列表中，为每个组合再次遍历整个内容并将字典合并到一个新字典中。谢谢！

Answer 1

首先，获取您要求的输出：

data = [{'name': "john", 'place': "nyc", 'owns': "gold", 'quantity': 30},
{'name': "john", 'place': "nyc", 'owns': "silver", 'quantity': 20},
{'name': "jane", 'place': "nyc", 'owns': "platinum", 'quantity': 5},
{'name': "john", 'place': "chicago", 'owns': "brass", 'quantity': 60},
{'name': "john", 'place': "chicago", 'owns': "silver", 'quantity': 40}]

from collections import defaultdict

accumulator = defaultdict(list)

for p in data:
    accumulator[p['name'],p['place']].append((p['owns'],p['quantity']))

from itertools import chain

[dict(chain([('name',name), ('place',place)], rest)) for (name,place),rest in accumulator.iteritems()]
Out[13]: 
[{'name': 'jane', 'place': 'nyc', 'platinum': 5},
 {'brass': 60, 'name': 'john', 'place': 'chicago', 'silver': 40},
 {'gold': 30, 'name': 'john', 'place': 'nyc', 'silver': 20}]

现在我不得不指出，您要求的这个字典列表数据结构非常尴尬。 dicts 非常适合 lookups，但是当您可以对整个对象组使用 one 时，它们的性能最好 - 如果您必须线性搜索一堆 dicts 来查找您想要的那个，您立即失去了dict 首先提供的全部好处。所以这给我们留下了几个选择。更深一层 - 将 dicts 嵌套在我们的 dict 中，或者完全使用其他东西。

我可以建议列出一个有意义的对象，每个对象代表这些人中的一个吗？要么创建自己的class，要么使用namedtuple：

from collections import namedtuple

Person = namedtuple('Person','name place holdings')

[Person(name, place, dict(rest)) for (name,place), rest in accumulator.iteritems()]
Out[17]: 
[Person(name='jane', place='nyc', holdings={'platinum': 5}),
 Person(name='john', place='chicago', holdings={'brass': 60, 'silver': 40}),
 Person(name='john', place='nyc', holdings={'silver': 20, 'gold': 30})]

Answer 2

因此，我的个人策略大致概述如下。您应该在给定字典实例的情况下定义一个密钥生成器，然后通过生成的该密钥将其分组到一个孤立的字典中。遍历所有元素并根据键进行更新后，只需返回分组字典的 .values()。

dicts = [
    {"name": "john", "place": "nyc", "owns": "gold", "quantity": 30},
    {"name": "john", "place": "nyc", "owns": "silver", "quantity": 20},
    {"name": "jane", "place": "nyc", "owns": "platinum", "quantity": 5},
    {"name": "john", "place": "chicago", "owns": "brass", "quantity": 60},
    {"name": "john", "place": "chicago", "owns": "silver", "quantity": 40}
]

def get_key(instance):
    return "%s-%s" % (instance.get("name"), instance.get("place"), )

grouped = {}

for dict_ in dicts:
    grouped[get_key(dict_)] = grouped.get(get_key(dict_), {})
    grouped[get_key(dict_)].update(dict_)

print grouped.values()
# [
#   {'owns': 'platinum', 'place': 'nyc', 'name': 'jane', 'quantity': 5},
#   {'name': 'john', 'place': 'nyc', 'owns': 'silver', 'quantity': 20}, 
#   {'name': 'john', 'place': 'chicago', 'owns': 'silver', 'quantity': 40}
# ]

Answer 3

这是一种方法：

dicts = [
    {"name": "john", "place": "nyc", "owns": "gold", "quantity": 30},
    {"name": "john", "place": "nyc", "owns": "silver", "quantity": 20},
    {"name": "jane", "place": "nyc", "owns": "platinum", "quantity": 5},
    {"name": "john", "place": "chicago", "owns": "brass", "quantity": 60},
    {"name": "john", "place": "chicago", "owns": "silver", "quantity": 40}
]

我们创建一个转换后的字典，以place-name 为键，输出字典为值

transformed_dict = {}
for a_dict in dicts:
    key = '{}-{}'.format(a_dict['place'], a_dict['name'])
    if key not in transformed_dict:
        transformed_dict[key] = {'name': a_dict['name'], 'place': a_dict['place'], a_dict['owns']: a_dict['quantity']}
    else:
        transformed_dict[key][a_dict['owns']] = a_dict['quantity']

transformed_dict 现在看起来像：

{'chicago-john': {'brass': 60,
                  'name': 'john',
                  'place': 'chicago',
                  'silver': 40},
 'nyc-jane': {'name': 'jane', 'place': 'nyc', 'platinum': 5},
 'nyc-john': {'gold': 30, 'name': 'john', 'place': 'nyc', 'silver': 20}}

pprint(list(transformed_dict.values())) 给出了我们想要的：

[{'gold': 30, 'name': 'john', 'place': 'nyc', 'silver': 20},
 {'brass': 60, 'name': 'john', 'place': 'chicago', 'silver': 40},
 {'name': 'jane', 'place': 'nyc', 'platinum': 5}]

Answer 4

from itertools import groupby
result, get_owns = [], lambda x: x["owns"]
get_details =  lambda x: (x["name"], x["place"])

# Sort and group the data based on name and place
for key, grp in groupby(sorted(data, key=get_details), key=get_details):

    # Create a dictionary with the name and place
    temp = dict(zip(("name", "place"), key))

    # Sort and group the grouped data based on owns
    for owns, grp1 in groupby(sorted(grp, key=get_owns), key=get_owns):

        # For each material, find and add the sum of quantity in temp
        temp[owns] = sum(item["quantity"] for item in grp1)

    # Add the temp dictionary to the result :-)
    result.append(temp)
print result

输出

[{'name': 'jane', 'place': 'nyc', 'platinum': 5},
 {'brass': 60, 'name': 'john', 'place': 'chicago', 'silver': 40},
 {'gold': 30, 'name': 'john', 'place': 'nyc', 'silver': 20}]

Answer 5

这可能是一个疯狂的想法，但是 dict-of-dicts-of-dicts 怎么样？这就像一个二维数组，行和列索引是名称和地点。

my_dicts = [
    {"name": "john", "place": "nyc", "owns": "gold", "quantity": 30},
    {"name": "john", "place": "nyc", "owns": "silver", "quantity": 20},
    {"name": "jane", "place": "nyc", "owns": "platinum", "quantity": 5},
    {"name": "john", "place": "chicago", "owns": "brass", "quantity": 60},
    {"name": "john", "place": "chicago", "owns": "silver", "quantity": 40}
]

all_names = set(d["name"] for d in my_dicts)
all_places = set(d["place"] for d in my_dicts)

merged = {name : {place : {} for place in all_places} for name in all_names}

for d in my_dicts:
    merged[d["name"]][d["place"]][d["owns"]] = d["quantity"]

import pprint
pprint.pprint(merged)

# {'jane': {'chicago': {}, 'nyc': {'platinum': 5}},
#  'john': {'chicago': {'brass': 60, 'silver': 40},
#           'nyc': {'gold': 30, 'silver': 20}}}

然后转换成你想要的格式：

new_dicts = [{"name" : name, "place" : place} for name in all_names for place in all_places if merged[name][place]]
for d in new_dicts:
    d.update(merged[d["name"]][d["place"]])
pprint.pprint(new_dicts)

# [{'name': 'jane', 'place': 'nyc', 'platinum': 5},
#  {'gold': 30, 'name': 'john', 'place': 'nyc', 'silver': 20},
#  {'brass': 60, 'name': 'john', 'place': 'chicago', 'silver': 40}]