从转换为 csv？答案

【问题标题】：Converting to csv from?从转换为 csv？
【发布时间】：2014-02-26 19:36:47
【问题描述】：

我有一个包含以下几行的文件

{"status":"OK","message":"OK","data":[{"type":"addressAccessType","addressAccessId":"0a3f508f-e7c8-32b8-e044-0003ba298018","municipalityCode":"0766","municipalityName":"Hedensted","streetCode":"0072","streetName":"Værnegården","streetBuildingIdentifier":"13","mailDeliverySublocationIdentifier":"","districtSubDivisionIdentifier":"","postCodeIdentifier":"8000","districtName":"Århus","presentationString":"Værnegården 13, 8000 Århus","addressSpecificCount":1,"validCoordinates":true,"geometryWkt":"POINT(553564 6179299)","x":553564,"y":6179299}]}

我想将每一行转换为带有标题的 csv 可读文件。像下面这样

status,message,data,addressAccessId,municipalityCode,municipalityName,streetCode,streetName,streetBuildingIdentifier,mailDeliverySublocationIdentifier,districtSubDivisionIdentifier,postCodeIdentifier,districtName,presentationString,addressSpecificCount,validCoordinates,geometryWkt,x,y
OK,OK,data:type,addressAccessType,0a3f508f-e7c8-32b8-e044-0003ba298018,0766,Hedensted,0072,Værnegården,13,,,8000,Århus,Værnegården 13, 8000 Århus,1,true,POINT553564 6179299,553564,6179299

我该如何做到这一点？代码和解释非常受欢迎。到目前为止，这就是我从这个例子中得出的以下内容：(How can I convert JSON to CSV?)

x = json.loads(x)

f = csv.writer(open('test.csv', 'wb+'))

# Write CSV Header, If you dont need that, remove this line
f.writerow(['status', 'message', 'type', 'addressAccessId', 'municipalityCode','municipalityName','streetCode','streetName','streetBuildingIdentifier','mailDeliverySublocationIdentifier','districtSubDivisionIdentifier','postCodeIdentifier','districtName','presentationString','addressSpecificCount','validCoordinates','geometryWkt','x','y'])


for x in x:
    f.writerow([x['status'], 
                x['message'], 
                x['data']['type'], 
                x['data']['addressAccessId'],
                x['data']['municipalityCode'],
                x['data']['municipalityName'],
                x['data']['streetCode'],
                x['data']['streetName'],
                x['data']['streetBuildingIdentifier'],
                x['data']['mailDeliverySublocationIdentifier'],
                x['data']['districtSubDivisionIdentifier'],
                x['data']['postCodeIdentifier'],
                x['data']['districtName'],
                x['data']['presentationString'],
                x['data']['addressSpecificCount'],
                x['data']['validCoordinates'],
                x['data']['geometryWkt'],
                x['data']['x'],
                x['data']['y']])

我已经查看并尝试了许多其他解决方案，包括 DictWriter、replace() 和 translate() 来删除字符，但还不能根据我的需要转换行。目的是能够选择输出到新文件中的字段，并将 x 和 y 转换为新的坐标系。但现在我只是试图将上面的行解析为 csv 文件。任何人都可以提供代码及其代码的解释吗？非常感谢您的宝贵时间。

下面是我的addresses.txt的前几行

    {"status":"OK","message":"OK","data":[{"type":"addressAccessType","addressAccessId":"0a3f5081-e039-32b8-e044-0003ba298018","municipalityCode":"0265","municipalityName":"Roskilde","streetCode":"0831","streetName":"Brønsager","streetBuildingIdentifier":"69","mailDeliverySublocationIdentifier":"","districtSubDivisionIdentifier":"Svogerslev","postCodeIdentifier":"4000","districtName":"Roskilde","presentationString":"Brønsager 69, 4000 Roskilde","addressSpecificCount":1,"validCoordinates":true,"geometryWkt":"POINT(690026 6169309)","x":690026,"y":6169309}]}
    {"status":"OK","message":"OK","data":[{"type":"addressAccessType","addressAccessId":"0a3f5089-ecab-32b8-e044-0003ba298018","municipalityCode":"0461","municipalityName":"Odense","streetCode":"9505","streetName":"Vægtens Kvarter","streetBuildingIdentifier":"271","mailDeliverySublocationIdentifier":"","districtSubDivisionIdentifier":"Holluf Pile","postCodeIdentifier":"5220","districtName":"Odense SØ","presentationString":"Vægtens Kvarter 271, 5220 Odense SØ","addressSpecificCount":1,"validCoordinates":true,"geometryWkt":"POINT(592191 6135829)","x":592191,"y":6135829}]}
    {"status":"OK","message":"OK","data":[{"type":"addressAccessType","addressAccessId":"0a3f507c-adc3-32b8-e044-0003ba298018","municipalityCode":"0165","municipalityName":"Albertslund","streetCode":"0445","streetName":"Skyttehusene","streetBuildingIdentifier":"33","mailDeliverySublocationIdentifier":"","districtSubDivisionIdentifier":"","postCodeIdentifier":"2620","districtName":"Albertslund","presentationString":"Skyttehusene 33, 2620 Albertslund","addressSpecificCount":1,"validCoordinates":true,"geometryWkt":"POINT(711079 6174741)","x":711079,"y":6174741}]}
    {"status":"OK","message":"OK","data":[{"type":"addressAccessType","addressAccessId":"0a3f509c-7f57-32b8-e044-0003ba298018","municipalityCode":"0851","municipalityName":"Aalborg","streetCode":"5205","streetName":"Løvstikkevej","streetBuildingIdentifier":"36","mailDeliverySublocationIdentifier":"","districtSubDivisionIdentifier":"","postCodeIdentifier":"9000","districtName":"Aalborg","presentationString":"Løvstikkevej 36, 9000 Aalborg","addressSpecificCount":1,"validCoordinates":true,"geometryWkt":"POINT(552407 6322490)","x":552407,"y":6322490}]}
    {"status":"OK","message":"OK","data":[{"type":"addressAccessType","addressAccessId":"0a3f5098-32a6-32b8-e044-0003ba298018","municipalityCode":"0779","municipalityName":"Skive","streetCode":"0462","streetName":"Landevejen","streetBuildingIdentifier":"52","mailDeliverySublocationIdentifier":"","districtSubDivisionIdentifier":"Håsum","postCodeIdentifier":"7860","districtName":"Spøttrup","presentationString":"Landevejen 52, 7860 Spøttrup","addressSpecificCount":1,"validCoordinates":true,"geometryWkt":"POINT(491515 6269739)","x":491515,"y":6269739}]}

【问题讨论】：

当您的列表也称为 x 时，我不会将 x 重复用于迭代变量。

标签： python json parsing csv nested

【解决方案1】：

请注意，data 键包含一个列表字典。 x['data']['type'] 不起作用，但 x['data'][0]['type'] 会。但是，该列表中可能有不止一本这样的字典。我假设您需要一个 CSV 行 每个 x['data'] 字典。

接下来，您似乎在每一行都有一个 UTF-8 BOM ;无论写什么，都没有正确使用 UTF-8 编码。我们需要去掉这个标记，前 3 个字符。

最后，JSON 字符串始终是 Unicode 数据，并且您的数据中包含非 ASCII 字符，因此在将数据传递给 CSV writer 对象之前，您必须再次编码为字节字符串。

我会在这里使用csv.DictWriter，并带有预定义的字段名称列表：

import codecs
import csv
import json

fields = [
    'status', 'message', 'type', 'addressAccessId', 'municipalityCode', 
    'municipalityName', 'streetCode', 'streetName', 'streetBuildingIdentifier',
    'mailDeliverySublocationIdentifier', 'districtSubDivisionIdentifier',
    'postCodeIdentifier', 'districtName', 'presentationString', 'addressSpecificCount',
    'validCoordinates', 'geometryWkt', 'x', 'y']


with open('test.csv', 'wb') as csvfile, open('jsonfile', 'r') as jsonfile:
    writer = csv.DictWriter(csvfile, fields)
    writer.writeheader()

    for line in jsonfile:
        if line.startswith(codecs.BOM_UTF8):
            line = line[3:]
        entry = json.loads(line)
        for item in entry['data']:
            row = dict(item, status=entry['status'], message=entry['message'])
            row = {k.encode('utf8'): unicode(v).encode('utf8') for k, v in row.iteritems()}
            writer.writerow(row)

row 字典基本上是 entry['data'] 列表中每个字典的副本，status 和 message 键分别复制。这使得 row 变成了一个平面字典。

我还逐行阅读了您的输入文件，正如您所说，每一行都包含一个单独的 JSON 条目。

【讨论】：

你想把writer.writerow(row) 放在for 循环中吗？
非常感谢您的详细回答，这绝对有很大帮助。您可以正确地假设我有一个包含多行的文件，其中我想要的数据位于 x[data] 中。但是，当我尝试您的代码时，我收到以下错误 ValueError: No JSON object could be decoded 是因为文件包含我的 json 行还是因为这些行不是有效的 json？
@JLLagrange：我确实有。
@PhilipHoyos：你的文件有空行吗？是所有 JSON 对象都在不同的行上，每行一个，还是有一个 big JSON 对象，它是一个对象列表，还是完全不同的格式。您能给我们提供前几行的示例吗？
@MartijnPieters 文件中的每一行都是相同的，并且具有相同的结构。它不是一个 big JSON 文件。我已将文件的前 4 行添加到问题中。感谢大家抽出宝贵的时间！

【解决方案2】：

使用 cvs.DictWriter() 打开输出文件并按照您的指定定义输出头字段。使用 extrasaction='ignore' 和 restval='' 作为选项。

查看Opening A large JSON file in Python with no newlines for csv conversion Python 2.6.6 以获取处理大文件的帮助，因为我有一个类似的问题也请查看我链接到的问题。

我使用适当的循环从 JSON 构建类似类型的系统。

例如，

def parse_row(currdata):
  outx = {}
  # currdata is defined earlier to point to the x['data'] dictionary
  for eachx in currdata:
    outx[eachx] = currdata[eachx]
  return outx

这是在以 currdata 作为参数并以 x['data'][row] 作为输入参数调用的函数中。

rows = len(x['data'])
for row in range(rows):
  outx = parse_row(x['data'][row])
  # process the row and create output

这应该允许您正确设置解析。我无法将实际代码复制到此答案中，但这应该为您指明解决方案。

【讨论】：