【问题标题】:Need help to extract list of dictionaries from Dataframe column需要帮助从 Dataframe 列中提取字典列表
【发布时间】:2021-06-23 21:31:04
【问题描述】:

    "cities":[69 items
    0:{10 items
    "name":"Autauga"
    "date":"2021-06-22"
    "fips":1001
    "lat":"32.53952745"
    "long":"-86.64408227"
    "confirmed":7242
    "deaths":113
    "confirmed_diff":0
    "deaths_diff":0
    "last_update":"2021-06-23 04:21:46"
    }
    1:{...}10 items
    2:{...}10 items
    3:{...}10 items
    4:{...}10 items
    5:{...}10 items

这是我的响应 JSON,带有城市标签,这是一个字典列表,这是我的代码

url = "https://covid-19-statistics.p.rapidapi.com/reports"
    
querystring = {"date":"2020-04-16","q":"US Alabama","region_name":"US","iso":"USA"}
#querystring = {"date":date,"iso":iso_code}
reports_response = requests.request("GET", url, headers=headers, params=querystring)
reports_resp = json.loads(reports_response.text)
#print(reports_resp['data'])
data_tuples=list(zip(reports_resp['data']))
        
df = pd.DataFrame(data_tuples,columns=['totals'])
df = pd.concat([df['totals'].apply(pd.Series), df.drop('totals', axis = 1)], axis = 1)
df = pd.concat([df['region'].apply(pd.Series), df.drop('region', axis = 1)], axis = 1)
df = pd.concat([df['cities'].apply(pd.Series), df.drop('cities', axis = 1)], axis = 1)
df = pd.concat([df[0].apply(pd.Series), df.drop(0, axis = 1)], axis = 1)

运行此代码后,我的 Dataframe 仅显示一个城市,但我的目标行为是创建与目标状态下的城市一样多的行。非常感谢任何帮助。

cities 列包含每个城市的词典列表,我想提取词典列表并为每个城市创建单独的行。

当我使用下面的代码时,我正在提取字典列表,但创建单独的列而不是行,并且它按照代码工作正常,但我想创建单独的行任何想法。


    df = pd.concat([df['cities'].apply(pd.Series), df.drop('cities', axis = 1)], axis = 1)

和

更新原始帖子,这是来自 API 的 JSON 响应

{1 item
"data":[1 item
0:{12 items
"date":"2021-06-22"
"confirmed":549013
"deaths":11311
"recovered":0
"confirmed_diff":0
"deaths_diff":0
"recovered_diff":0
"last_update":"2021-06-23 04:21:46"
"active":537702
"active_diff":0
"fatality_rate":0.0206
"region":{6 items
"iso":"USA"
"name":"US"
"province":"Alabama"
"lat":"32.3182"
"long":"-86.9023"
"cities":[69 items
0:{10 items
"name":"Autauga"
"date":"2021-06-22"
"fips":1001
"lat":"32.53952745"
"long":"-86.64408227"
"confirmed":7242
"deaths":113
"confirmed_diff":0
"deaths_diff":0
"last_update":"2021-06-23 04:21:46"
}
1:{...}10 items
2:{...}10 items
3:{...}10 items
4:{...}10 items
5:{...}10 items
6:{...}10 items
7:{...}10 items
8:{...}10 items
9:{...}10 items
10:{...}10 items
11:{...}10 items
12:{...}10 items
13:{...}10 items
14:{...}10 items
15:{...}10 items
16:{...}10 items
17:{...}10 items
18:{...}10 items
19:{...}10 items
20:{...}10 items
21:{...}10 items
22:{...}10 items
23:{...}10 items
24:{...}10 items
25:{...}10 items
26:{...}10 items
27:{...}10 items
28:{...}10 items
29:{...}10 items
30:{...}10 items
31:{...}10 items
32:{...}10 items
33:{...}10 items
34:{...}10 items
35:{...}10 items
36:{...}10 items
37:{...}10 items
38:{...}10 items
39:{...}10 items
40:{...}10 items
41:{...}10 items
42:{...}10 items
43:{...}10 items
44:{...}10 items
45:{...}10 items
46:{...}10 items
47:{...}10 items
48:{...}10 items
49:{...}10 items
50:{...}10 items
51:{...}10 items
52:{...}10 items
53:{...}10 items
54:{...}10 items
55:{...}10 items
56:{...}10 items
57:{...}10 items
58:{...}10 items
59:{...}10 items
60:{...}10 items
61:{...}10 items
62:{...}10 items
63:{...}10 items
64:{...}10 items
65:{...}10 items
66:{...}10 items
67:{...}10 items
68:{...}10 items
]
}
}
]
}

【问题讨论】:

  • 您能否提供headers 变量或至少reports_resp 值?谢谢。
  • 我认为 headers 包含 API 密钥,因此来自 reports_resp 的示例会很棒。
  • 使用来自 API 的完整 JSON 响应更新原始帖子

标签: python pandas dataframe data-science


【解决方案1】:

通过您提供的示例找到解决方案并不容易,但我做了一些假设,稍后将详细说明。我合并了您作为服务输出提供的字典中 "data" 键下提供的所有信息。您需要处理我的解决方案中的DataFrame 以满足您的需求。

import pandas as pd

# Use json_normalize to flat JSON response
dfj = pd.json_normalize(data_tuples["data"])
region_cities = []  # Auxiliary
# Here the magic lies in apply(pd.json_normalize)
#  since you want to normalize the cities JSON structure of each row
for rc in dfj["region.cities"].apply(pd.json_normalize).iteritems():
    # Assign the index of the original DataFrame as a new column to keep reference
    region_cities.append(rc[1].assign(index=[rc[0]]*len(rc[1].index)))
# Join original DataFrame with nested cities structure DataFrame
# The reference of the index added in a previous step is used to join
dfj = dfj.join(pd.concat(region_cities).set_index("index"), rsuffix="_region.cities")
# Cosmetic changes: drop the "region.cities" column and reset the index
dfj.drop("region.cities", axis=1).reset_index(drop=True)

这是我使用的结构(我在您提供的服务输出中添加了一些假值):

data_tuples = {
   "data":[
      {
         "date":"2021-06-22",
         "confirmed":549013,
         "deaths":11311,
         "recovered":0,
         "confirmed_diff":0,
         "deaths_diff":0,
         "recovered_diff":0,
         "last_update":"2021-06-23 04:21:46",
         "active":537702,
         "active_diff":0,
         "fatality_rate":0.0206,
         "region":{
            "iso":"USA",
            "name":"US",
            "province":"Alabama",
            "lat":"32.3182",
            "long":"-86.9023",
            "cities":[
               {
                  "name":"Autauga",
                  "date":"2021-06-22",
                  "fips":1001,
                  "lat":"32.53952745",
                  "long":"-86.64408227",
                  "confirmed":7242,
                  "deaths":113,
                  "confirmed_diff":0,
                  "deaths_diff":0,
                  "last_update":"2021-06-23 04:21:46"
               },
               {
                  "name":"Autauga2",
                  "date":"2021-06-22",
                  "fips":1002,
                  "lat":"33.53952745",
                  "long":"-87.64408227",
                  "confirmed":7243,
                  "deaths":114,
                  "confirmed_diff":1,
                  "deaths_diff":1,
                  "last_update":"2021-06-23 04:21:43"
               }
            ]
         }
      },
      {
         "date":"2021-06-23",
         "confirmed":549014,
         "deaths":11312,
         "recovered":1,
         "confirmed_diff":0,
         "deaths_diff":0,
         "recovered_diff":0,
         "last_update":"2021-06-24 04:21:46",
         "active":537702,
         "active_diff":0,
         "fatality_rate":0.0206,
         "region":{
            "iso":"USA2",
            "name":"US2",
            "province":"Alabama2",
            "lat":"38.3182",
            "long":"-88.9023",
            "cities":[
               {
                  "name":"Autauga3",
                  "date":"2021-06-22",
                  "fips":1001,
                  "lat":"32.53952745",
                  "long":"-86.64408227",
                  "confirmed":7242,
                  "deaths":113,
                  "confirmed_diff":0,
                  "deaths_diff":0,
                  "last_update":"2021-06-23 04:21:46"
               },
               {
                  "name":"Autauga4",
                  "date":"2021-06-22",
                  "fips":1002,
                  "lat":"33.53952745",
                  "long":"-87.64408227",
                  "confirmed":7243,
                  "deaths":114,
                  "confirmed_diff":1,
                  "deaths_diff":1,
                  "last_update":"2021-06-23 04:21:43"
               }
            ]
         }
      }
   ]
}

【讨论】:

    猜你喜欢
    • 1970-01-01
    • 1970-01-01
    • 2021-09-21
    • 2021-09-21
    • 1970-01-01
    • 2021-09-22
    • 1970-01-01
    • 2010-11-14
    • 1970-01-01
    相关资源
    最近更新 更多