将 CSV 文件转换为 JSON 文件

Question

我正在尝试将我的 CSV 文件转换为 JSON 文件格式。当我这样做时，JSON 文件中有一个额外的条目，它只包含字段名称。

我尝试过使用 pandas、字典，但似乎无法得到结果。某事或其他事来了。

我想删除 JSON 开头的仅额外文件名条目。另外，我怎样才能使 ConnectionId 作为键，并为不同的输出设置相同的格式。

import csv, json

csvfile = open('/home/Desktop/PD/GEOSubscriberLocations_LTE_sample.csv', 'r')
jsonfile = open('/home/Desktop/PD/script5.json', 'w')

fieldnames = ("Confidence", "ConnectionId", "Imei", "Imsi", "IsData", "IsSignalling", "IsVoice", "Latitude", "Longitude",
              "Mcc", "Mnc", "SegmentDuration", "SegmentStartTime", "ServingCellLabel", "Sv", 
              "TrackingAreaCode", "Uncertainity")

reader = csv.DictReader(csvfile , fieldnames)

code = ''
for row in reader:
    for key in row:
        row[key] = row[key].decode('utf-8', 'ignore').encode('utf-8')
        json.dump(row, jsonfile, indent=4, sort_keys=False)
        jsonfile.write('\n')

实际结果是：

{
    "Confidence": "Confidence", 
    "IsData": "IsData", 
    "Latitude": "Latitude", 
    "ConnectionId": "ConnectionId", 
    "Mcc": "Mcc", 
    "Sv": "Sv", 
    "Longitude": "Longitude", 
    "Uncertainity": "Uncertainty", 
    "IsVoice": "IsVoice", 
    "IsSignalling": "IsSignalling", 
    "SegmentStartTime": "SegmentStartTime", 
    "Imei": "Imei", 
    "SegmentDuration": "SegmentDuration", 
    "Mnc": "Mnc", 
    "ServingCellLabel": "ServingCellLabel", 
    "Imsi": "Imsi", 
    "TrackingAreaCode": "TrackingAreaCode"
}
{
    "Confidence": "1.994667E-07", 
    "IsData": "FALSE", 
    "Latitude": "1.694202", 
    "ConnectionId": "330708186825281", 
    "Mcc": "999", 
    "Sv": "01", 
    "Longitude": "0.434623", 
    "Uncertainity": "178", 
    "IsVoice": "FALSE", 
    "IsSignalling": "TRUE", 
    "SegmentStartTime": "16/02/2017 09:56:59.912", 
    "Imei": "99999006686069", 
    "SegmentDuration": "00:00:00.0350000", 
    "Mnc": "99", 
    "ServingCellLabel": "Cell18", 
    "Imsi": "999992223223602", 
    "TrackingAreaCode": "1234"
}
{
    "Confidence": "1.504506E-12", 
    "IsData": "FALSE", 
    "Latitude": "1.633704", 
    "ConnectionId": "260339442647675", 
    "Mcc": "999", 
    "Sv": "02", 
    "Longitude": "0.668554", 
    "Uncertainity": "314", 
    "IsVoice": "FALSE", 
    "IsSignalling": "TRUE", 
    "SegmentStartTime": "16/02/2017 09:57:01.377", 
    "Imei": "99999207564306", 
    "SegmentDuration": "00:00:00.0280000", 
    "Mnc": "99", 
    "ServingCellLabel": "Cell19", 
    "Imsi": "999993793410366", 
    "TrackingAreaCode": "1235"
}
{
    "Confidence": "0.3303348", 
    "IsData": "FALSE", 
    "Latitude": "1.847635", 
    "ConnectionId": "260339442647676", 
    "Mcc": "999", 
    "Sv": "14", 
    "Longitude": "1.356349", 
    "Uncertainity": "129", 
    "IsVoice": "FALSE", 
    "IsSignalling": "TRUE", 
    "SegmentStartTime": "16/02/2017 09:57:01.555", 
    "Imei": "99999605176135", 
    "SegmentDuration": "00:00:00.0290000", 
    "Mnc": "99", 
    "ServingCellLabel": "Cell13", 
    "Imsi": "999992216631694", 
    "TrackingAreaCode": "1236"
}
{
    "Confidence": "0.01800376", 
    "IsData": "FALSE", 
    "Latitude": "1.914598", 
    "ConnectionId": "330708186825331", 
    "Mcc": "999", 
    "Sv": "74", 
    "Longitude": "1.222736", 
    "Uncertainity": "463", 
    "IsVoice": "FALSE", 
    "IsSignalling": "TRUE", 
    "SegmentStartTime": "16/02/2017 09:57:02.689", 
    "Imei": "99999007880884", 
    "SegmentDuration": "00:00:00.0260000", 
    "Mnc": "99", 
    "ServingCellLabel": "Cell7", 
    "Imsi": "999992226681236", 
    "TrackingAreaCode": "1237"
}
{
    "Confidence": "0.2068138", 
    "IsData": "FALSE", 
    "Latitude": "1.850279", 
    "ConnectionId": "330708186825354", 
    "Mcc": "999", 
    "Sv": "13", 
    "Longitude": "1.349263", 
    "Uncertainity": "167", 
    "IsVoice": "FALSE", 
    "IsSignalling": "TRUE", 
    "SegmentStartTime": "16/02/2017 09:57:04.351", 
    "Imei": "99999002855874", 
    "SegmentDuration": "00:00:00.0300000", 
    "Mnc": "99", 
    "ServingCellLabel": "Cell15", 
    "Imsi": "999995430231562", 
    "TrackingAreaCode": "1238"
}

如果使用 ConnectionId 作为键，我希望我的输出如下：

{
    "ConnectionId": "189970698469977",
        {
            "Confidence": "0.01428183",
            "Imei": "99999507405260",
            "Imsi": "999992226504812",
            "IsData": "FALSE",
            "IsSignalling": "TRUE",
            "IsVoice": "FALSE",
            "Latitude": "1.848613",
            "Longitude": "1.354355",
            "Mcc": "999",
            "Mnc": "99",
            "SegmentDuration": "00:00:00.0860000",
            "SegmentStartTime": "16/02/2017 09:57:00.053",
            "ServingCellLabel": "Cell14",
            "Sv": "06",
            "TrackingAreaCode": "1256",
            "Uncertainty": 662
        }

Answer 1

尝试用以下代码替换您的 for 循环：

arr = []

with open (csvFile) as f:
    csvReader = csv.DictReader(f)
    #print(csvReader)
    for csvRow in csvReader:
        arr.append(csvRow)

print(arr)

# write the data to a json file
with open(jsonFile, "w") as jsonFile:
    jsonFile.write(json.dumps(arr, indent = 4))

也请参考这个link。

Answer 2

额外的仅字段名称条目

如果您明确提供字段名，csv 将假定 .csv 文件的第一行是数据。如果省略 fieldnames 参数，它将假定 .csv 文件的第一行是带有字段名称的标题行：

fieldnames 参数是一个序列。如果省略了fieldnames，文件f第一行的值将作为fieldnames。

看起来您的 .csv 文件有一个标题行，但您还明确提供了字段名，因此 csv 已将标题行作为数据读取。要仅使用标题行中的字段名，请将您对 DictReader 的调用更改为：

csv.DictReader(csvfile)  # notice no fieldnames parameter

使用某个字段作为键

首先考虑如何最好地在 JSON 中表示这一点，以及您试图从该字段索引中获得什么，您提供的示例不是很有效的 JSON。

{
    "ConnectionId": "189970698469977",
        {
            "Confidence": "0.01428183",
            "Imei": "99999507405260",
            ...
        }

这是无效的，因为：

• 我们打开一个{，表示这是一个“对象”
• 对象具有键和与这些键关联的值，没有其他任何东西
• 我们为它提供一个键“ConnectionID”和一个值。这很好
• 然后我们提供另一个对象但没有键，这是无效的。

假设您希望能够根据 connectionId 快速查找对象，我们如何在 JSON 中创建一个如下所示的对象：

{
    "189970698469977": {
        "Confidence": "0.01428183",
        "Imei": "99999507405260",
        ...
    },
    "260339442647676": {
        "Confidence": ...
    },
    ...
}

这为我们提供了一种令人满意的属性，即 JSON 仅在键唯一时才有效。

为此，我们需要在 Python 中创建一个字典，我们将在 JSON 转储中表示：

我们可以从(key, value) 元组序列中创建 Python 字典。来自the docs的示例：

>>> dict([('sape', 4139), ('guido', 4127), ('jack', 4098)])
{'sape': 4139, 'guido': 4127, 'jack': 4098}

我们将使用这个构造函数来创建我们的索引字典：

dictionaryEntries = [(row['ConnectionId'], row) for row in csvReader]
dictionaryToDump = dict(dictionaryEntries)

把它放在一起

您的代码现在可能如下所示：

import csv
import json

with open('mycsv.csv') as csvFile:
  csvReader = csv.DictReader(csvFile)
  dictionaryEntries = [(row['ConnectionId'], row) for row in csvReader]

dictionaryToDump = dict(dictionaryEntries)

with open('myjson.json', 'w') as jsonFile:
    jsonFile.write(json.dumps(dictionaryToDump))