Python pandas json_normalize 加入数据框字段

Question

我有一个数据框：

df = pd.DataFrame([
    [1, '{"issues": [{"issue_name": "fixed.issue.cpeUnreachable", "issue_id": "52446*", "actions": [], "issueFixed": "true"}, {"issue_name": "fixed.issue.internet.cgnat.statusActive", "issueFixed": "false", "issue_id": "8834*4", "actions": [            {"action_name": "cableCheck", "success": "false"},             {"action_name": "otherCheck", "success": "true"}]}, {"issue_name": "fixed.issue.rf.ds.quality", "issue_id": "3642*", "actions": [            {"action_name": "akcija 1", "success": "false"},             {"action_name": "akcija 2", "success": "false"},             {"action_name": "akcija 3", "success": "false"},             {"action_name": "akcija 4", "success": "false"},             {"action_name": "akcija 5", "success": "false"}], "issueFixed": "true"}, {"issue_name": "fixed.issue.rf.us.quality", "issueFixed": "false", "issue_id": "8834*3", "actions": []},    {"issue_name" : "rebootBeforeTicket",        "actions" : [{"action_name": "rebootCpeDevice", "success" : "false"},            {"action_name": "rebootStbDevice", "success" : "true"}]} ]}'],
    [2, '{"issues": [{"issue_name": "fixed.issue.cpeUnreachable", "issue_id": "52446*", "actions": [], "issueFixed": "true"}, {"issue_name": "fixed.issue.internet.cgnat.statusActive", "issueFixed": "false", "issue_id": "8834*4", "actions": [            {"action_name": "cableCheck", "success": "false"},             {"action_name": "otherCheck", "success": "true"}]}, {"issue_name": "fixed.issue.rf.ds.quality", "issue_id": "3642*", "actions": [            {"action_name": "akcija 1", "success": "false"},             {"action_name": "akcija 2", "success": "false"},             {"action_name": "akcija 3", "success": "false"},             {"action_name": "akcija 4", "success": "false"},             {"action_name": "akcija 5", "success": "false"}], "issueFixed": "true"}, {"issue_name": "fixed.issue.rf.us.quality", "issueFixed": "false", "issue_id": "8834*3", "actions": []},    {"issue_name" : "rebootBeforeTicket",        "actions" : [{"action_name": "rebootCpeDevice", "success" : "false"},            {"action_name": "rebootStbDevice", "success" : "true"}]} ]}']], 
    columns=['session_id', 'json_text'])
df

我想将此数据框转换为：

到目前为止，我已经尝试了以下方法：

df1 = pd.DataFrame()

for idx, row in df.iterrows():
    json_contents = json.loads(row.stat_dimen_value)
    df_json = json_normalize(json_contents['issues'], record_path=['actions'], meta=['issue_id', 'issue_name', 'issueFixed'], errors='ignore')
    df_json.insert(0, 'session_id', row.session_id)
    df1 = pd.concat([df1, df_json])


df1 = df1[['session_id', 'issue_id', 'issue_name', 'issueFixed', 'action_name', 'success']]

它有效，但我对 for 循环不满意。 我必须将新创建的 df_json（来自 df.json_text 字段）数据框与 df.session_id 字段一起加入。由于找不到其他方法，我使用了 for 循环。

有没有更好的方法在不使用 for 循环的情况下将 df_json 与其 df.session_id（可能还有其他 df 字段）字段连接起来？

问候。

编辑1，json注入解决方案：

json_ser = df.apply(lambda row: json.loads(row.stat_dimen_value[:1] + f'"session_id":{row.session_id}, ' + row.stat_dimen_value[1:]), axis=1)
json_ser.head()


df1 = json_normalize(json_ser, \
    record_path=['issues', 'actions'], \
    meta=['session_id', ['issues', 'issue_id'], ['issues', 'issue_name'], ['issues', 'issueFixed']], \
    sep='_', \
    errors='ignore') \
    .rename(columns={'issues_issue_id' : 'issue_id', 'issues_issue_name' : 'issue_name', 'issues_issueFixed' : 'issueFixed'}) \
    [['session_id', 'issue_id', 'issue_name', 'issueFixed', 'action_name', 'success']]

apply 将字段 session_id 注入到 json 中，并且 json_normalize 具有所有用于解析的信息。

我创建了包含 2048 行的测试性能数据框。在我的笔记本电脑上，for 循环耗时 8.92 秒，而 apply + json_normalize 耗时 387 + 167 毫秒。 看起来注入json要快得多。

Answer 1

我能做的最好的就是这个 - 仍然使用循环，但肯定会为您节省一些时间

def joiner(x):
    a = pd.DataFrame(json_normalize(json.loads(x[1])['issues'], record_path=['actions'], 
        meta=['issue_id','issue_name','issueFixed'],errors='ignore'))
    return pd.concat([pd.DataFrame({'Session':[x[0]] * a.shape[0]}), a], axis=1)

pd.concat([joiner(x[1] )for x in df.iterrows()])