将来自 API 服务的嵌套 JSON 响应解析为 python 中的 csv

Question

我正在尝试以清晰有序的方式将 API 响应的输出保存到 CSV 文件中，这是检索 API 数据的脚本：

import json
import requests
import csv

# List of keywords to be checked
keywords = open("/test.txt", encoding="ISO-8859-1")

keywords_to_check = []

try:
    for keyword in keywords:
        keyword = keyword.replace("\n", "")
        keywords_to_check.append(keyword)
except Exception:
        print("An error occurred. I will try again!")
        pass

apikey = # my api key
apiurl = # api url
apiparams = {
    'apikey': apikey, 
    'keyword': json.dumps(keywords_to_check), 
    'metrics_location': '2840',
    'metrics_language': 'en',
    'metrics_network': 'googlesearchnetwork',
    'metrics_currency': 'USD',
    'output': 'csv'
}
response = requests.post(apiurl, data=apiparams)
jsonize = json.dumps(response.json(), indent=4, sort_keys=True)

if response.status_code == 200:
    print(json.dumps(response.json(), indent=4, sort_keys=True))

我得到的输出如下：

{
    "results": {
        "bin": {
            "cmp": 0.795286539,
            "cpc": 3.645033,
            "m1": 110000,
            "m10": 90500,
            "m10_month": 2,
            "m10_year": 2019,
            "m11": 135000,
            "m11_month": 1,
            "m11_year": 2019,
            "m12": 135000,
            "m12_month": 12,
            "m12_year": 2018,
            "m1_month": 11,
            "m1_year": 2019,
            "m2": 110000,
            "m2_month": 10,
            "m2_year": 2019,
            "m3": 110000,
            "m3_month": 9,
            "m3_year": 2019,
            "m4": 135000,
            "m4_month": 8,
            "m4_year": 2019,
            "m5": 135000,
            "m5_month": 7,
            "m5_year": 2019,
            "m6": 110000,
            "m6_month": 6,
            "m6_year": 2019,
            "m7": 110000,
            "m7_month": 5,
            "m7_year": 2019,
            "m8": 90500,
            "m8_month": 4,
            "m8_year": 2019,
            "m9": 90500,
            "m9_month": 3,
            "m9_year": 2019,
            "string": "bin",
            "volume": 110000
        },
        "chair": {
            "cmp": 1,
            "cpc": 1.751945,
            "m1": 1000000,
            "m10": 823000,
            "m10_month": 2,
            "m10_year": 2019,
            "m11": 1500000,
            "m11_month": 1,
            "m11_year": 2019,
            "m12": 1500000,
            "m12_month": 12,
            "m12_year": 2018,
            "m1_month": 11,
            "m1_year": 2019,
            "m2": 1000000,
            "m2_month": 10,
            "m2_year": 2019,
            "m3": 1000000,
            "m3_month": 9,
            "m3_year": 2019,
            "m4": 1220000,
            "m4_month": 8,
            "m4_year": 2019,
            "m5": 1220000,
            "m5_month": 7,
            "m5_year": 2019,
            "m6": 1000000,
            "m6_month": 6,
            "m6_year": 2019,
            "m7": 1000000,
            "m7_month": 5,
            "m7_year": 2019,
            "m8": 1000000,
            "m8_month": 4,
            "m8_year": 2019,
            "m9": 1000000,
            "m9_month": 3,
            "m9_year": 2019,
            "string": "chair",
            "volume": 1220000
        }, ....

我想要实现的是一个显示以下信息和排序的 csv 文件，列是字符串、cmp、cpc 和体积：

字符串；cmp；cpc；音量
仓;0.795286539;3.645033;110000
椅子;1;1.751945;1220000

根据 Sidous 的建议，我得出以下结论：

import pandas as pd
data = response.json()
df = pd.DataFrame.from_dict(data)
df.head()

哪个游戏给我以下输出：

结果
bin {'string': 'bin', 'volume': 110000, 'm1': 1100...
椅子{'字符串'：'椅子'，'音量'：1220000，'m1'：1 ...
花 {'string': 'flower', 'volume': 1830000, 'm1': ...
表 {'string': 'table', 'volume': 673000, 'm1': 82...
水{'字符串'：'水'，'体积'：673000，'m1'：67 ...

关闭，但我怎样才能将“字符串”、“音量”等显示为列并避免显示字典的 {？

非常感谢谁能帮我解决这个问题:)

歪斜

Answer 1

我建议将响应保存在 pandas 数据框中，然后由 pandas 存储（你知道 csv 文件很容易被 pandas 处理）。

import pandas as pd


# receiving results in a dictionary
dic = response.json()

# remove the results key from the dictionary
dic = dic.pop("results", None)

# convert dictionary to dataframe
data = pd.DataFrame.from_dict(dic, orient='index')

# string;cmp;cpc;volume
new_data = pd.concat([data['string'], data['cmp'], data['cpc'], data['volume']], axis=1)

# removing the default index (bin and chair keys)
new_data.reset_index(drop=True, inplace=True)

print(new_data)

# saving new_data into a csv file
new_data.to_csv('name_of_file.csv')

你在python文件的同一目录下找到csv文件（否则你可以在.to_csv()方法中指定）。

您可以在下面的屏幕截图中看到最终结果。

Answer 2

使用with open 命令打开一个文本文件，并通过遍历整个dict 进一步写下数据

with open("text.csv", "w+") as f:
    f.write('string;cmp;cpc;volume\n')
    for res in response.values():     #This is after I assumed that `response` is of type dict
        for r in res.values():
            f.write(r['string']+';'+str(r['cmp'])+';'+str(r['cpc'])+';'+str(r['volume'])+'\n')

Answer 3

试试这个：

import pandas as pd

data = response.json()
cleaned_data = []

for key, val in data["results"].items():
    cleaned_data.append(val)

df = pd.DataFrame.from_dict(cleaned_data)
df1 = df[["string","cmp","cpc","volume"]]
df1.head()
df1.to_csv("output.csv")

Answer 4

使用csv.DictWriter 怎么样，因为您的数据几乎就是它所需要的功能？

import csv

if __name__ is "__main__":
  results = {"chair": {"cmp": 1, "cpc": 3.64}, "bin": {"cmp": 0.5, "cpc": 1.75}} # two rows will do for the example
  # Now let's get the data structure we really want: a list of rows
  rows = []
  for key, value in results:
    rows.append(results)
    # And, while we're at it, set the string part
    rows[-1]["string"] = key

  # Create the header
  fieldnames = set()
  for row in rows:
    for fname in row:
      fieldnames.add(fname)

  # Write to the file
  with open("mycsv.csv", "w", newline="") as file_:
    writer = csv.DictWriter(file_, fieldnames=fieldnames)
    writer.writeheader()
    for row in rows:
      writer.writerow(row)

你应该擅长这种东西，而不使用任何其他库