【问题标题】:what is the efficient way to convert json file to dataframe?将json文件转换为数据框的有效方法是什么?
【发布时间】:2018-05-12 12:59:12
【问题描述】:

我有一个json文件格式:

[
    {
      "2018-05-11 09:45:10": {
        "change": 6.4,
        "change rate": 1.58,
        "code": "00700",
        "current": 412.6,
        "market_cap": 39212.21,
        "turnover": 419550479.8,
        "volume": 102009800.0
      }
    },
    {
      "2018-05-11 09:45:20": {
        "change": 6.8,
        "change rate": 1.67,
        "code": "00700",
        "current": 413.0,
        "market_cap": 39250.23,
        "turnover": 493879299.8,
        "volume": 120029800.0
        }
    },    ... ]

我想把json文件转成dataframe格式:

       time   code  current  change  change rate  market_cap  \
0   2018-05-11 09:45:10  00700    412.6     6.4         1.58    39212.21   
1   2018-05-11 09:45:20  00700    413.0     6.8         1.67    39250.23   
2   2018-05-11 09:45:30  00700    413.2     7.0         1.72    39269.23   
3   2018-05-11 09:45:40  00700    413.0     6.8         1.67    39250.23   
4   2018-05-11 09:45:50  00700    413.0     6.8         1.67    39250.23
...

        turnover       volume  
0   4.195505e+08  102009800.0  
1   4.938793e+08  120029800.0  
2   5.581315e+08  135588900.0  
3   5.804374e+08  140989900.0  
4   5.956777e+08  144679900.0  
...

这是我的代码:

def convert_json_file_to_df(file_path):    
    with open(file_path, encoding='utf-8') as fh:
        jd = json.load(fh, encoding='utf-8')

    col_list = ["time", "code", "current", "change", "change rate", "market_cap", "turnover", "volume"]
    df = pd.DataFrame(columns=col_list)

    for d in jd:
        for key, value in d.items():
            df = df.append({"time": key,
                            "code": value["code"],
                            "current": value["current"],
                            "change": value["change"],
                            "change rate": value["change rate"],
                            "market_cap": value["market_cap"],
                            "turnover": value["turnover"],
                            "volume": value["volume"]
                            }, ignore_index=True)

    print(df)

我想有一种简短有效的方法将 json 文件转换为数据帧。我写的代码,我认为它很慢而且不好看。有没有更有效的方法?另一个问题是如何以 dict 格式附加 json 文件?非常感谢

更新: 追加json文件的代码

def save_dict_to_json_file(dict, filepath):
    if ((type(dict)!=type({})) or (not dict) or (not filepath)):
        return FALSE

    try:
        with open(filepath, encoding='utf-8') as f:
            json_data = json.load(f, encoding='utf-8')

        json_data.append(dict)

        with open(filepath, mode='w', encoding='utf-8') as f:
            json.dump(json_data, f, ensure_ascii=False, indent=2, sort_keys=True)

        return TRUE

    except Exception as e:
        traceback.print_exc()
        err = sys.exc_info()[1]

        return FALSE, str(err)

【问题讨论】:

  • 你能解释更多The other question is how can I append the json file in dict format?吗?
  • @jezrael 现在我使用 [ ] 来存储 json,当我使用 json.load 读取文件时,输出的类型是 。我尝试使用 { } replace [ ],但它会读取文件:json.decoder.JSONDecodeError: Expecting property name enclosed in double quotes: line 2 column 5 (char 6)
  • 你创建 json 吗?如果是这样,创建一个像{date1:{},date2:{}} 这样的文件(一个字典的字典,其中第一个字典的标签是索引值(日期),第二个字典的标签是你现在拥有的列)会将问题减少到@ 987654329@
  • @shadowdk - 没有文件很难知道,数据是否保密?
  • @jezrael 我只是更新了我用来附加 json 文件的代码

标签: python-3.x dataframe


【解决方案1】:

我认为您可以使用 list comprehension 和扁平化嵌套字典、add 新元素 time 和最后一个 DataFrame 构造函数:

L = [dict(v, time=k) for x in jd for k, v in x.items()]
print (L)
[{'change': 6.4, 'change rate': 1.58, 'code': '00700', 'current': 412.6, 
  'market_cap': 39212.21, 'turnover': 419550479.8, 'volume': 102009800.0, 
  'time': '2018-05-11 09:45:10'}, 
 {'change': 6.8, 'change rate': 1.67, 'code': '00700', 'current': 413.0, 
  'market_cap': 39250.23, 'turnover': 493879299.8, 'volume': 120029800.0,
  'time': '2018-05-11 09:45:20'}]

col_list = ["time", "code", "current", "change", 
            "change rate", "market_cap", "turnover", "volume"]
df = pd.DataFrame(L, columns=col_list)
print (df)
                  time   code  current  change  change rate  market_cap  \
0  2018-05-11 09:45:10  00700    412.6     6.4         1.58    39212.21   
1  2018-05-11 09:45:20  00700    413.0     6.8         1.67    39250.23   

      turnover       volume  
0  419550479.8  102009800.0  
1  493879299.8  120029800.0  

【讨论】:

    猜你喜欢
    • 2011-05-29
    • 2020-01-15
    • 1970-01-01
    • 2021-05-22
    • 1970-01-01
    • 2011-05-20
    • 1970-01-01
    • 1970-01-01
    • 2014-08-01
    相关资源
    最近更新 更多