将 NBA 比赛按比赛特定的 .json 转换为 .csv

Question

我正在尝试为我的理学硕士建立一个数据库，其中包含几个赛季 NBA 比赛的逐场比赛数据。在经济学论文中。目前我正在从 NBA 的 API (see example) 中提取游戏，并使用 this routine 将每个游戏拆分为不同的 .json 文件（适用于 p-b-p 目的），从而生成 .json 文件（第一个播放示例）：

{"headers": ["GAME_ID", "EVENTNUM", "EVENTMSGTYPE", "EVENTMSGACTIONTYPE", "PERIOD", "WCTIMESTRING", "PCTIMESTRING", "HOMEDESCRIPTION", "NEUTRALDESCRIPTION", "VISITORDESCRIPTION", "SCORE", "SCOREMARGIN"], "rowSet": [["0041400406", 0, 12, 0, 1, "9:11 PM", "12:00", null, null, null, null, null], ["0041400406", 1, 10, 0, 1, "9:11 PM", "12:00", "Jump Ball Mozgov vs. Green: Tip to Barnes", null, null, null, null]

我计划创建一个循环以将所有生成的 .json 文件转换为 .csv，这样我就可以继续在 stata 中进行计量经济分析。目前，我被困在这个过程的第一步：创建 json 到 CSV 的转换过程（我会在之后设计循环）。我正在尝试的代码是：

f = open('pbp_0041400406.json') 
data = json.load(f) 
f.close()

with open("pbp_0041400406.csv", "w") as file:
    csv_file = csv.writer(file)

    for rowSet in data:
        csv_file.writerow(rowSet)

f.close()

但是，生成的 CSV 文件显示出尴尬的结果：一行读取 h,e,a,d,e,r,s，另一行读取 r,o,w,S,e,t，因此无法捕获标题或 rowSet（戏剧本身）。

考虑到on this thread 的贡献，我已经尝试解决这个问题，但我一直没能做到。谁能给我一些解决这个问题的见解？

[编辑] 用原始代码中的数据替换行集也产生了相同的结果。

提前致谢！

Answer 1

试试这个：

import json
import csv

with open('json.json') as f:
    data = json.load(f)


with open("pbp_0041400406.csv", "w") as fout:
    csv_file = csv.writer(fout, quotechar='"')

    csv_file.writerow(data['headers'])

    for rowSet in data['rowSet']:
        csv_file.writerow(rowSet)

生成的 CSV：

GAME_ID,EVENTNUM,EVENTMSGTYPE,EVENTMSGACTIONTYPE,PERIOD,WCTIMESTRING,PCTIMESTRING,HOMEDESCRIPTION,NEUTRALDESCRIPTION,VISITORDESCRIPTION,SCORE,SCOREMARGIN

0041400406,0,12,0,1,9:11 PM,12:00,,,,,

0041400406,1,10,0,1,9:11 PM,12:00,Jump Ball Mozgov vs. Green: Tip to Barnes,,,,

Answer 2

我认为您可能对 json 输入的结构有误。顶层有三个键。 resultSets -> 列表，其第一个元素是带有键“rowSet”的字典。这就是我认为你想要迭代的内容。

f = open('playbyplay', 'r')
data = json.load(f)
f.close()
print data.keys()
rows = data['resultSets'][0]['rowSet']

with open("pbp_0041400406.csv", "w") as file:
    csv_file = csv.writer(file)
    for rowSet in rows:
       csv_file.writerow(rowSet) 

输出数据：

0041300402,0,12,0,1,8:13 PM,12:00,,,,,,0,0,,,,,,0,0,,,,,,0,0,,,,,
0041300402,1,10,0,1,8:13 PM,12:00,Jump Ball Duncan vs. Bosh: Tip to Wade,,,,,4,1495,Tim Duncan,1610612759,San Antonio,Spurs,SAS,5,2547,Chris Bosh,1610612748,Miami,Heat,MIA,5,2548,Dwyane Wade,1610612748,Miami,Heat,MIA
0041300402,2,5,2,1,8:13 PM,11:45,Green STEAL (1 STL),,James Lost Ball Turnover (P1.T1),,,5,2544,LeBron James,1610612748,Miami,Heat,MIA,4,201980,Danny Green,1610612759,San Antonio,Spurs,SAS,0,0,,,,,
0041300402,3,1,1,1,8:14 PM,11:26,Green 18' Jump Shot (2 PTS) (Splitter 1 AST),,,0 - 2,2,4,201980,Danny Green,1610612759,San Antonio,Spurs,SAS,4,201168,Tiago Splitter,1610612759,San Antonio,Spurs,SAS,0,0,,,,,
0041300402,4,6,2,1,8:14 PM,11:03,Green S.FOUL (P1.T1),,,,,4,201980,Danny Green,1610612759,San Antonio,Spurs,SAS,5,1740,Rashard Lewis,1610612748,Miami,Heat,MIA,1,0,,,,,
0041300402,5,3,11,1,8:14 PM,11:03,,,Lewis Free Throw 1 of 2 (1 PTS),1 - 2,1,5,1740,Rashard Lewis,1610612748,Miami,Heat,MIA,0,0,,,,,,0,0,,,,,
0041300402,7,3,12,1,8:14 PM,11:03,,,MISS Lewis Free Throw 2 of 2,,,5,1740,Rashard Lewis,1610612748,Miami,Heat,MIA,0,0,,,,,,0,0,,,,,
0041300402,9,4,0,1,8:15 PM,11:01,Splitter REBOUND (Off:0 Def:1),,,,,4,201168,Tiago Splitter,1610612759,San Antonio,Spurs,SAS,0,0,,,,,,0,0,,,,,
0041300402,10,1,5,1,8:15 PM,10:52,Duncan 1' Layup (2 PTS) (Parker 1 AST),,,1 - 4,3,4,1495,Tim Duncan,1610612759,San Antonio,Spurs,SAS,4,2225,Tony Parker,1610612759,San Antonio,Spurs,SAS,0,0,,,,,

Answer 3

问题解决了！使用@MaxU 代码和先前构建的包含所有gameid 的.CSV，可以通过.JSON 直接抓取自01-02 赛季以来的每场nba 比赛，并使用以下代码转换为CSV：（@MaxU 的致谢）

from __future__ import print_function

import json
import csv
import requests

u_a = "Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/48.0.2564.82 Safari/537.36"
url_pattern = "http://stats.nba.com/stats/playbyplayv2?GameID=%(GameID)s&StartPeriod=%(StartPeriod)s&EndPeriod=%(EndPeriod)s&tabView=%(tabView)s"


def write_csv(game_id, resultSet):
fn = resultSet['name'] + '_' + str(game_id) + '.csv'
# ignore unimportant resultsets ...
if resultSet['name'] not in ['PlayByPlay', 'PlayBlahBlah']:
    return
with open(fn, 'w') as fout:
    csv_file = csv.writer(fout, quotechar='"')

    csv_file.writerow(resultSet['headers'])

    for rowSet in resultSet['rowSet']:
        csv_file.writerow(rowSet)

def process_game_id(game_id, tabView='playbyplay',
                start_period='0', end_period='0'):
url_parms = {
    'GameID': game_id,
    'StartPeriod': start_period,
    'EndPeriod': end_period,
    'tabView': tabView,
}

r = requests.get((url_pattern % url_parms), headers={"USER-AGENT":u_a})

if r.status_code == requests.codes.ok:
    data = json.loads(r.text)

    for rset in data['resultSets']:
        write_csv(url_parms['GameID'], rset)
else:
    r.raise_for_status()


if __name__ == '__main__':
#
# assuming that the 'games.csv' file contains all Game_IDs ...
#
with open('games.csv', 'r') as f:
    csv_reader = csv.reader(f, delimiter=',')
    for row in csv_reader:
        process_game_id(row[<column_num_containing_Game_ID>])

对此数据有任何其他问题，请私信我。祝大家编码愉快！