如何循环不同字典的值并将它们附加到熊猫数据框中

Question

我需要循环不同字典的值并从中创建一个数据框。

数据来自一个输出json的api，如下所示

源字典

result = {
    "meta": {
        "request": {
            "segment_name": "Searches1",
            "metrics": ["Visits"]
        },
        "status": "Success"
    },
    "segments": [
        {"date": "2021-11-01", "visits": 100, "confidence": "High"},
        {"date": "2021-11-02", "visits": 200, "confidence": "High"},
        {"date": "2021-11-03", "visits": 300, "confidence": "Low"},
        {"date": "2021-11-04", "visits": 400, "confidence": "High"},
        {"date": "2021-11-05", "visits": 500, "confidence": "Low"},
    ]
},
{
    "meta": {
        "request": {
            "segment_name": "Searches2",
            "metrics": ["Visits"]
        },
        "status": "Success"
    },
    "segments": [
        {"date": "2021-11-01", "visits": 110, "confidence": "High"},
        {"date": "2021-11-02", "visits": 220, "confidence": "High"},
        {"date": "2021-11-03", "visits": 330, "confidence": "Low"},
        {"date": "2021-11-04", "visits": 440, "confidence": "High"},
        {"date": "2021-11-05", "visits": 540, "confidence": "Low"},
    ]
}

我尝试了以下方法，我只是循环“segments”-dictionairy，但这显然不起作用。

我的方法

def getSearches():

    Searches = []
    segment_name = result['meta']['request']['segment_name']

    if "segments" in result:
        for fs in result['segments']:
            Searches.append(
                {"date": fs['date'], "segment_name": segment_name, "visits": fs['visits'], "confidence": fs['confidence']})

    fs_df = pd.DataFrame(Searches)
    print(fs_df)


getSearches()

我收到以下错误消息

错误信息

Traceback (most recent call last):
  File "/Users/ismail/Desktop/sw_dict_test", line 51, in <module>
    getFlightSearches()
  File "/Users/ismail/Desktop/sw_dict_test", line 40, in getFlightSearches
    segment_name = result['meta']['request']['segment_name']
TypeError: tuple indices must be integers or slices, not str

确切地说，我需要从“request”字典中访问“segment_name”以及“segments”字典中的所有变量，并将它们附加到 pandas 表中。

期望的输出

         date segment_name  visits confidence
0  2021-11-01    Searches1     100       High
1  2021-11-02    Searches1     200       High
2  2021-11-03    Searches1     300        Low
3  2021-11-04    Searches1     400       High
4  2021-11-05    Searches1     500        Low
5  2021-11-01    Searches2     110       High
6  2021-11-02    Searches2     220       High
7  2021-11-03    Searches2     330        Low
8  2021-11-04    Searches2     440       High
9  2021-11-05    Searches2     550        Low

我怎样才能做到这一点？

Answer 1

您还可以使用json_normalize 来展平 JSON 数据。由于记录列表，即您需要转换为行的字典存储在“段”中，因此设置record_path='segments'。您只使用“segment_name”作为每条记录的元数据，因此您将其路径设置为列表：meta=[['meta', 'request', 'segment_name']]。

然后使用rename 更改列名并使用reindex 以正确顺序获取列。

df = pd.json_normalize(result, 'segments', [['meta', 'request', 'segment_name']]).rename({'meta.request.segment_name':'segment_name'}, axis=1).reindex(['date', 'segment_name', 'visits', 'confidence'], axis=1)

输出：

         date segment_name  visits confidence
0  2021-11-01    Searches1     100       High
1  2021-11-02    Searches1     200       High
2  2021-11-03    Searches1     300        Low
3  2021-11-04    Searches1     400       High
4  2021-11-05    Searches1     500        Low
5  2021-11-01    Searches2     110       High
6  2021-11-02    Searches2     220       High
7  2021-11-03    Searches2     330        Low
8  2021-11-04    Searches2     440       High
9  2021-11-05    Searches2     540        Low

Answer 2

result 是一个元组，因此错误。将其改为列表并遍历每个元素。

result = [{
    "meta": {
        "request": {
            "segment_name": "Searches1",
            "metrics": ["Visits"]
        },
        "status": "Success"
    },
    "segments": [
        {"date": "2021-11-01", "visits": 100, "confidence": "High"},
        {"date": "2021-11-02", "visits": 200, "confidence": "High"},
        {"date": "2021-11-03", "visits": 300, "confidence": "Low"},
        {"date": "2021-11-04", "visits": 400, "confidence": "High"},
        {"date": "2021-11-05", "visits": 500, "confidence": "Low"},
    ]
},
{
    "meta": {
        "request": {
            "segment_name": "Searches2",
            "metrics": ["Visits"]
        },
        "status": "Success"
    },
    "segments": [
        {"date": "2021-11-01", "visits": 110, "confidence": "High"},
        {"date": "2021-11-02", "visits": 220, "confidence": "High"},
        {"date": "2021-11-03", "visits": 330, "confidence": "Low"},
        {"date": "2021-11-04", "visits": 440, "confidence": "High"},
        {"date": "2021-11-05", "visits": 540, "confidence": "Low"},
    ]
}]


def getSearches(result):

    Searches = []
    segment_name = result['meta']['request']['segment_name']

    if "segments" in result:
        for fs in result['segments']:
            Searches.append(
                {"date": fs['date'], "segment_name": segment_name, "visits": fs['visits'], "confidence": fs['confidence']})

    return Searches

searches = []
for r in result:
    searches += getSearches(r)
    
pd.DataFrame(searches)