将 JSON 转换为 pandas DataFrame- Python

Question

我已使用以下 Python 行从特定 API 读取数据

import requests 
import json
# read all Measurement from one sensor for several days.
r = requests.get('https://wastemanagement.post-iot.lu/measurement/measurements?source=83512 pageSize=1000000000&dateFrom=2019-10-26&dateTo=2019-10-28', auth=('xxxxx', 'xxxxx'))
text=r.text  # data is stored in json format  

这是一个输出示例，但实际输出包含几行： 实际输出可以从这里下载actual output

enter code here
Valid JSON (RFC 8259)
Formatted JSON Data
{
"next":"https://wastemanagement.post-iot.lu/measurement/measurements?dateTo=2019-10-28&   
 pageSize=2000&source=83512&dateFrom=2019-10-26&currentPage=2",
"self":"https://wastemanagement.post-iot.lu/measurement/measurements?dateTo=2019-10-28& 
 pageSize=2000&source=83512&dateFrom=2019-10-26&currentPage=1",
"statistics":{
"totalPages":null,
"currentPage":1,
"pageSize":2000
},
"measurements":[
{
     "self":"https://wastemanagement.post-iot.lu/measurement/measurements/108451",
     "time":"2019-10-26T00:00:06.494Z",
     "id":"108451",
     "source":{
        "self":"https://wastemanagement.post-iot.lu/inventory/managedObjects/83512",
        "id":"83512"
     },
     "type":"c8y_Measurement",
     "battery":{
        "percent":{
           "unit":"%",
           "value":98
        }
     }
  },
  {
     "self":"https://wastemanagement.post-iot.lu/measurement/measurements/108452",
     "time":"2019-10-26T00:00:06.538Z",
     "id":"108452",
     "source":{
        "self":"https://wastemanagement.post-iot.lu/inventory/managedObjects/83512",
        "id":"83512"
     },
     "type":"TemperatureMeasurement",
     "c8y_TemperatureMeasurement":{
        "T":{
           "unit":"C",
           "value":23
        }
     }
  },
  {
     "self":"https://wastemanagement.post-iot.lu/measurement/measurements/108537",
     "time":"2019-10-26T00:00:06.577Z",
     "id":"108537",
     "source":{
        "self":"https://wastemanagement.post-iot.lu/inventory/managedObjects/83512",
        "id":"83512"
     },
     "type":"c8y_Measurement",
     "c8y_DistanceMeasurement":{
        "distance":{
           "unit":"cm",
           "value":21
        }
     }
  },

所附图片显示（文本）中的输出

我尝试使用以下代码将 JSON 格式（存储在文本中）转换为 pandas DataFrame，但不起作用

data = json.loads(text) 

我想将 JSON 格式转换为 pandas DataFrame，此 DataFrame 应包含 五列（id、电池、T 或温度、时间和距离）和 几行

Answer 1

你可以试试这个，效果很好

// importing required libraries
import pandas as pd
import json
import requests

// hosted your json response as a url response 
URL = 'https://my-json-server.typicode.com/abhikumar22/JsonServer/data'

// getting requests from the server
req = requests.get(URL )
text_data= req.text
json_dict= json.loads(text_data)

// converting the json dictionary to a dataframe
df = pd.DataFrame.from_dict(json_dict["measurements"])
cols_to_keep = ['id','battery','c8y_TemperatureMeasurement','time','c8y_DistanceMeasurement']
df_final = df[cols_to_keep]
df_final = df_final.rename(columns={'c8y_TemperatureMeasurement': 'Temperature Or T','c8y_DistanceMeasurement':'Distance'})
print(df_final)

result of the code you can see find here

您将获得所需的结果，其中包含一些列值，您可以进一步修改以获取列值。

Answer 2

这应该可行。顺便说一句，我不确定您在数据框中具体需要哪个时间。因此，我将它们全部包含在解决方案中（如果您不确定测量的顺序）

import pandas as pd
import numpy as np
import json
from collections import OrderedDict
json_str = {
"next":"https://wastemanagement.post-iot.lu/measurement/measurements?dateTo=2019-10-28&pageSize=2000&source=83512&dateFrom=2019-10-26&currentPage=2",
"self":"https://wastemanagement.post-iot.lu/measurement/measurements?dateTo=2019-10-28&pageSize=2000&source=83512&dateFrom=2019-10-26&currentPage=1",
"statistics":{
"totalPages":"null",
"currentPage":1,
"pageSize":2000
},
"measurements":[
{
     "self":"https://wastemanagement.post-iot.lu/measurement/measurements/108451",
     "time":"2019-10-26T00:00:06.494Z",
     "id":"108451",
     "source":{
        "self":"https://wastemanagement.post-iot.lu/inventory/managedObjects/83512",
        "id":"83512"
     },
     "type":"c8y_Measurement",
     "battery":{
        "percent":{
           "unit":"%",
           "value":98
        }
     }
  },
  {
     "self":"https://wastemanagement.post-iot.lu/measurement/measurements/108452",
     "time":"2019-10-26T00:00:06.538Z",
     "id":"108452",
     "source":{
        "self":"https://wastemanagement.post-iot.lu/inventory/managedObjects/83512",
        "id":"83512"
     },
     "type":"TemperatureMeasurement",
     "c8y_TemperatureMeasurement":{
        "T":{
           "unit":"C",
           "value":23
        }
     }
  },
  {
     "self":"https://wastemanagement.post-iot.lu/measurement/measurements/108537",
     "time":"2019-10-26T00:00:06.577Z",
     "id":"108537",
     "source":{
        "self":"https://wastemanagement.post-iot.lu/inventory/managedObjects/83512",
        "id":"83512"
     },
     "type":"c8y_Measurement",
     "c8y_DistanceMeasurement":{
        "distance":{
           "unit":"cm",
           "value":21
        }
     }
   }]
     }


#json_str2 = json.dumps(dct)
df = pd.io.json.json_normalize(json_str)
df2 = pd.io.json.json_normalize(
    OrderedDict([(str(i), v) for i, v in enumerate(df["measurements"].tolist()[0])]))

# If you are certain that the list always comes in that order
df = pd.concat([df, df2], axis=1)
df[["0.source.id", "2.time","0.battery.percent.value", "1.c8y_TemperatureMeasurement.T.value", "2.c8y_DistanceMeasurement.distance.value"]]

# If you are uncertain of the order
cols = ['0.source.id'] + \
[c for c in df.columns if ('time' in c or 'emperatureMeasurement.T.value' in c or 'DistanceMeasurement.distance.unit' in c or 'battery.percent.value' in c)]
df[cols].head()