将 URL 中的结果存储到 Pandas Dataframe 中

Question

我是 Pandas 和 Python 的新手。有一个要求在哪里..

我正在使用 for 循环将 100 个邮政编码传递给一个 URL，并尝试提取每个传递的邮政编码的纬度和经度。 它的结果我需要保存在数据框中。下面是我正在使用的代码。

query_cust = "select custMasterID,Full_Name,POSTCODE from DMON.BANK_CUSTOMERS"
df_cust = pd.read_sql(query_cust, con=con_str)

df_cust["URL"] = "https://api.getthedata.com/postcode/" + df_cust['POSTCODE'].str.replace(" ", "")

for column in df_cust["URL"]:
    # print(column)
    response = requests.get(column)
    response_text = response.text
    #df = json.loads(response_text)['data']
    parse_json = json.loads(response_text)
    df_cust["Lat"] = pd.json_normalize(parse_json['data']['latitude'])
    df_cust["Long"] = parse_json['data']['longitude']
print(df_cust)

下面是我尝试运行它时出现的错误。

df_cust["Lat"] = pd.json_normalize(parse_json['data']['latitude'])

在 _json_normalize 引发 NotImplementedError 未实现错误

Answer 1

您无需使用json_normalize 从响应数据中获取您需要的信息。只需遍历数据框的每一行并更新值：

import pandas as pd
import json
import requests

pd.options.display.max_columns = None
pd.options.display.max_rows = None

df_cust = pd.DataFrame(columns=['POSTCODE'])

# Just appending some data
df_cust = df_cust.append({'POSTCODE': 'SW1A-1AA'}, ignore_index=True)
df_cust = df_cust.append({'POSTCODE': 'WC2B-4AB'}, ignore_index=True)
df_cust = df_cust.append({'POSTCODE': 'ASDF-QWE'}, ignore_index=True) # Wrong postal code

for i, row in df_cust.iterrows():
    df_cust.at[i, 'URL'] = 'https://api.getthedata.com/postcode/' + row['POSTCODE'].replace('-','')
    response = requests.get(df_cust.loc[i, 'URL'])
    parse_json = json.loads(response.text)
    
    if 'data' in parse_json:
        if 'latitude' in parse_json['data']:
            df_cust.at[i, 'LAT'] = parse_json['data']['latitude']
        else:
            df_cust.at[i, 'LAT'] = None
        if 'longitude' in parse_json['data']:
            df_cust.at[i, 'LON'] = parse_json['data']['longitude']
        else:
            df_cust.at[i, 'LON'] = None  
    else:
        df_cust.at[i, 'LAT'] = None
        df_cust.at[i, 'LON'] = None
    
print(df_cust)

输出：

   POSTCODE                                          URL        LAT        LON
0  SW1A-1AA  https://api.getthedata.com/postcode/SW1A1AA  51.501009  -0.141588
1  WC2B-4AB  https://api.getthedata.com/postcode/WC2B4AB  51.514206  -0.119893
2  ASDF-QWE  https://api.getthedata.com/postcode/ASDFQWE       None       None