找出两个 csv 文件之间的差异并标记 em

Question

我正在使用 Django 制作一个网站，该网站允许用户上传 2 个 csv 文件，然后网站会找到这两个文件之间的差异，打印差异并添加一个新的“更改”列。在每个记录的新列中，每一行都会获得一个新的“更改”或“添加”或“删除”新记录。假设没有重复。这是目前为止的views.py代码。

from django.shortcuts import render
from django.http import HttpResponse
import difflib
import datetime
import csv
from django.http import HttpResponseRedirect
from django.http import FileResponse
from .forms import FileForm
from .forms import UploadFileForm

def handle_uploaded_file(filename_1, filename_2):
    """ handle_uploaded_file is a function that takes 2 files uploaded by the users """

    with open(filename_1, newline='') as f_old:
        csv_old = csv.reader(f_old, delimiter='\t')
        header = next(csv_old)
        old_data = {row[0] : row for row in csv_old}

    with open(filename_2, newline='') as f_new:
        csv_new = csv.reader(f_new, delimiter='\t')
        header = next(csv_new)
        new_data = {row[0] : row for row in csv_new}

    set_new_data = set(new_data)
    set_old_data = set(old_data)

    added = [['Added'] + new_data[v] for v in set_new_data - set_old_data]
    deleted = [['Deleted'] + old_data[v] for v in set_old_data - set_new_data]
    in_both = set_old_data & set_new_data
    changed = [['Changed'] + new_data[v] for v in in_both if old_data[v] != new_data[v]]

    with open('difference.csv', 'w', newline='') as f_output:
        csv_output = csv.writer(f_output, delimiter='\t')
        csv_output.writerow(['History'] + header)
        csv_output.writerows(sorted(added + deleted + changed, key=lambda x: x[1:]))

def index(request): # index is a function for the upload button
    if request.method == 'POST': # POST method inserts something to the server
        print(request.FILES)
        form = UploadFileForm(request.POST, request.FILES)
        print(form.errors)
        if form.is_valid():
            print("cool")
            handle_uploaded_file(request.FILES.get('file1'),request.FILES.get('file2'))
            return HttpResponseRedirect('results/')
    else:
        form = UploadFileForm()
    return render(request, 'hello.html', {'form': form})

def results(request): # results is a function that sends difference.csv back to the user once the file is ready
    file_path = (r'C:\Users\Public\Documents\PycharmProjects\filecomparison\difference.csv') #  adding an absolute path in the server, pinpoints that exact file, very important, r is to produce raw string and handle unicodeescape error
    response = FileResponse(open(file_path, 'rb'))
    response['Content-Type'] = 'text/csv' # the type of the file that will be send is .txt/.csv
    response['Content-Disposition'] = 'attachment; filename=difference.csv' # produces an attachment file for users to download called with difference in .csv file
    return response

这是旧的.csv

Name    Emp_ID  Zone_ID Zone Name   Customer Type   First Name  Cust_ID Balance Loan    Asset Name  Serial  Purchase_ID 
Jack    4145    10-34-2Z-71ABD      Bakery  Matt    41235   -123.567    123.44  43521_BLACK_BURNER  52+007XX    QOLO-LW09   
Bob 4146    10-35-2Z-71ABD      Woodsmith   Dylan   12315   -129.12 194.44  43521_BLACK_BURNER  50+001XX    KDFL-LW10   
Rick    4147    10-34-2Z-73ABD      Gunrange    Robin   64234   -134.12 133.44  43521_BLACK_BURNER  32+003XX    HFDO-LW11   
Shane   4148    10-34-2Z-72ABD      Restaurant  Aldo    31356   -121.23 949.44  43521_BLACK_BURNER  89+004XX    BDSM-LW12   

这是新的.csv

Name    Emp_ID  Zone_ID Zone Name   Customer Type   First Name  Cust_ID Balance Loan    Asset Name  Serial  Purchase_ID
Jack    4145    10-34-2Z-71ABD      Bakery  Matt    41235   -123.567    123.44  43521_BLACK_BURNER  52+007XX    QOLO-LW09
Bob 4146    10-35-2Z-71ABD      Woodsmith   Ron 12315   -129.12 194.44  43521_BLACK_BURNER  50+001XX    KDFL-LW10
Jane    1234    19-35-2K-72XYZ      EO  Karen   50980   -547.95 544.39  43521_BLACK_BURNER  50+0076KK   ERQW-BN66
Shane   4148    10-34-2Z-72ABD      Restaurant  Aldo    31356   -121.23 949.44  43521_BLACK_BURNER  89+004XX    BDSM-LW12

这应该是差异.csv

Name    Emp_ID  Zone_ID Zone Name   Customer Type   First Name  Cust_ID Balance Loan    Asset Name  Serial  Purchase_ID Changes
Bob 4146    10-35-2Z-71ABD      Woodsmith   Ron 12315   -129.12 194.44  43521_BLACK_BURNER  50+001XX    KDFL-LW10   Changed
Jane    1234    19-35-2K-72XYZ      EO  Karen   50980   -547.95 544.39  43521_BLACK_BURNER  50+0076KK   ERQW-BN66   Added
Rick    4147    10-34-2Z-73ABD      Gunrange    Robin   64234   -134.12 133.44  43521_BLACK_BURNER  32+003XX    HFDO-LW11   Deleted

如果我运行 Martin Evan 的代码 ，这就是我得到的结果

目前，我收到“TypeError: expected str, bytes, or os.PathLike object, not list。”当我将我拥有的代码与他的代码混合时出错。但是，如果我单独运行 Martin 的代码，我会得到如下图所示的 difference.csv，我猜 delimiter='\t' 是“历史”和“列”混淆的原因。我试图添加 2 个分隔符，希望它能将它们分开，但 python 不允许我这样做。另一方面，在谷歌搜索 TypeError 的解决方案后，我认为在 handle_uploaded_file 函数下使用 open() 方法可以解决问题，但它仍然存在，或者它可能不是正确的解决方案。非常感谢任何帮助

Answer 1

假设您不需要以某种方式处理old.csv 包含重复条目或new.csv 包含重复条目（即同一个名称在同一个文件中出现多次）的情况，那么正如建议的那样，一个实现这一点的方法是利用 Python 的集合。

通过从每个文件中创建一组名称，然后使用集合操作很容易确定添加、删除和更改。例如：

import csv

with open('old.csv', newline='') as f_old:
    csv_old = csv.reader(f_old, delimiter='\t')
    header = next(csv_old)
    old_data = {row[0] : row for row in csv_old}

with open('new.csv', newline='') as f_new:
    csv_new = csv.reader(f_new, delimiter='\t')
    header = next(csv_new)
    new_data = {row[0] : row for row in csv_new}

set_new_data = set(new_data)
set_old_data = set(old_data)

added = [['Added'] + new_data[v] for v in set_new_data - set_old_data]
deleted = [['Deleted'] + old_data[v] for v in set_old_data - set_new_data]
in_both = set_old_data & set_new_data
changed = [['Changed'] + new_data[v] for v in in_both if old_data[v] != new_data[v]]

with open('difference.csv', 'w', newline='') as f_output:
    csv_output = csv.writer(f_output, delimiter='\t')
    csv_output.writerow(['History'] + header)
    csv_output.writerows(sorted(added + deleted + changed, key=lambda x: x[1:]))

这会比较整行的差异。在 Python 3.6.3 上测试

---

尝试将此脚本添加到代码中的一个问题是 Python 3.x 中的 CSV 库需要使用 newline='' 参数打开文件（如果使用 2.x，则为二进制文件）。由于您的函数已经打开文件句柄，因此这是一个问题。请尝试以下操作：

import csv

def handle_uploaded_file(file_1, file_2): 
    """ handle_uploaded_file is a function that takes 2 files uploaded by the users """

    csv_old = csv.reader(file_1, delimiter='\t')
    header = next(csv_old)
    old_data = {row[0] : row for row in csv_old}

    csv_new = csv.reader(file_2, delimiter='\t')
    header = next(csv_new)
    new_data = {row[0] : row for row in csv_new}

    set_new_data = set(new_data)
    set_old_data = set(old_data)

    added = [['Added'] + new_data[v] for v in set_new_data - set_old_data]
    deleted = [['Deleted'] + old_data[v] for v in set_old_data - set_new_data]
    in_both = set_old_data & set_new_data
    changed = [['Changed'] + new_data[v] for v in in_both if old_data[v] != new_data[v]]

    with open('difference.csv', 'w', newline='') as f_output:
        csv_output = csv.writer(f_output, delimiter='\t')
        csv_output.writerow(['History'] + header)
        csv_output.writerows(sorted(added + deleted + changed, key=lambda x: x[1:]))