python：使用序列号编写 CSV 文件

Question

我正在尝试读取 csv 文件并根据其中的前 2 列写入内容，并带有序列号。这一切都很好..但我的挑战是附加文件而不是为每一行编写新文件..

test.csv

2002,2402,1,1,1,1,1,1,1,1
2002,2002,2,2,2,2,2,2,2,2
2402,2402,3,3,3,3,3,3,3,3
2002,2402,4,4,4,4,4,4,4,4
3333,2402,5,5,5,5,5,5,5,5
3333,3333,6,6,6,6,6,6,6,6

输出文件：

2002_200418_000000001.csv
1,1,1,1,1,1,1,1
2002_200418_000000002.csv
2,2,2,2,2,2,2,2
2002_200418_000000003.csv
4,4,4,4,4,4,4,4
2402_200418_000000001.csv
1,1,1,1,1,1,1,1
2402_200418_000000002.csv
3,3,3,3,3,3,3,3
2402_200418_000000003.csv
4,4,4,4,4,4,4,4
2402_200418_000000004.csv
5,5,5,5,5,5,5,5
3333_200418_000000001.csv
5,5,5,5,5,5,5,5
3333_200418_000000002.csv
6,6,6,6,6,6,6,6

python code:

import os, csv, datetime
from itertools import cycle

seq = {'2002': cycle(range(1,999)),'2402': cycle(range(1,999)),'3333': cycle(range(1,999))}

def SequenceNum(f):
    return f'{next(seq[f])}'.zfill(9)

def filegenerate(X):
    with open(X) as csvfile:
        timestamp = '_' + '{:%y%m%d%H%M%S}'.format(datetime.datetime.now()) + '_'
        reader = csv.reader(csvfile, quotechar="'")
        for cdr in reader:
            if cdr[0] == '' and cdr[1] == '':
                pass
            elif cdr[0] == cdr[1]:
                with open(cdr[0] + timestamp + SequenceNum(cdr[0]) + '.csv', 'a', newline='') as f:
                    writer = csv.writer(f, quotechar="'")
                    writer.writerow(cdr[2:])
                    f.close()
            elif cdr[0] != cdr[1] and cdr[1] != '' and cdr[0] != '':
                with open(cdr[0] + timestamp + SequenceNum(cdr[0]) + '.csv', 'a', newline='') as f:
                    writer = csv.writer(f, quotechar="'")
                    writer.writerow(cdr[2:])
                    f.close()
                with open(cdr[1] + timestamp + SequenceNum(cdr[1]) + '.csv', 'a', newline='') as f:
                    writer = csv.writer(f, quotechar="'")
                    writer.writerow(cdr[2:])
                    f.close()
            elif cdr[1] != '' and cdr[0] == '':
                with open(cdr[1] + timestamp + SequenceNum(cdr[1]) + '.csv', 'a', newline='') as f:
                    writer = csv.writer(f, quotechar="'")
                    writer.writerow(cdr[2:])
                    f.close()
            elif cdr[0] != '' and cdr[1] == '':
                with open(cdr[0] + timestamp + SequenceNum(cdr[0]) + '.csv', 'a', newline='') as f:
                    writer = csv.writer(f, quotechar="'")
                    writer.writerow(cdr[2:])
                    f.close()

if __name__ == '__main__':
    filegenerate('test.csv')

从上面的输出中，2002 文件根据行数生成了多次，我想将行附加到相应的文件中。请帮助改进我的代码。

expected output:

2002_200418_000000001.csv
1,1,1,1,1,1,1,1
2,2,2,2,2,2,2,2
4,4,4,4,4,4,4,4

2402_200418_000000001.csv
1,1,1,1,1,1,1,1
3,3,3,3,3,3,3,3
4,4,4,4,4,4,4,4
5,5,5,5,5,5,5,5

3333_200418_000000001.csv
5,5,5,5,5,5,5,5
6,6,6,6,6,6,6,6

我认为这个问题是因为在写入文件时分配了 seq 编号，可能我可以将所有行附加到 2002..etc 文件中，然后用 seq 编号重命名它们。但是有更好的方法吗？？

对于提出的为什么需要 seq 的问题，当每个文件以 000000001 结尾时，这是因为当我遍历多个 .csv 文件时，seq 编号将被分配并进一步增加。

example:
test1.csv
test2.csv
test3.csv

output:
2002_200418_000000001.csv
2402_200418_000000001.csv
3333_200418_000000001.csv

2002_200418_000000002.csv
2402_200418_000000002.csv
3333_200418_000000002.csv

2002_200418_000000003.csv
2402_200418_000000003.csv
3333_200418_000000003.csv

我的想法，seq 编号应该分配给每个 .csv，而不是分配给 csv 文件的每一行。 请帮忙...

提前致谢。

Answer 1

IIUC，用途：

def SequenceNum(seq):
    return f'{seq}'.zfill(9)

def write_file(n, data, seq):
    filename = n + '_' + datetime.datetime.now().strftime("%y%m%d") + '_' + SequenceNum(seq) + '.csv'
    with open(filename, 'a', newline='') as f:
        writer = csv.writer(f, quotechar="'")
        writer.writerow(data)

def read_files():
    files = ["test1.csv", "test2.csv", "test3.csv"]

    for idx, filename in enumerate(files, 1): # process each file one by one
        with open(filename, "r") as file:
            reader = csv.reader(file)
            for line in reader:
                n1, n2, *data = line
                write_file(n1, data, idx)
                if n1 != n2:
                    write_file(n2, data, idx)

调用函数：

read_files()

这将创建以下文件：

2002_200418_000000001.csv
2402_200418_000000001.csv
3333_200418_000000001.csv

2002_200418_000000002.csv
2402_200418_000000002.csv
3333_200418_000000002.csv

2002_200418_000000003.csv
2402_200418_000000003.csv
3333_200418_000000003.csv

Answer 2

这是一种在将未来文件内容写入磁盘之前将其收集到字典中的方法。对于每个未来的文件，在字典中创建一个条目，其中包含文件行的集合。我选择了一个集合，因为我假设如果您在 >= 3 的列中遇到相同的内容，您只想在输出中写入一次。如果不是这样，您可以将集合替换为列表。

import os, csv, datetime
from itertools import cycle

seq = { '2002': cycle(range(1,999)),
        '2402': cycle(range(1,999)),
        '3333': cycle(range(1,999)) }

def SequenceNum(f):
    return f'{next(seq[f])}'.zfill(9)

def filegenerate(filePath: str) -> None:
    with open(filePath, 'r') as csvFile:
        rows = [row for row in csv.reader(csvFile, quotechar="'")]

    fileContents = dict()
    for row in rows:
        fileContents.setdefault(row[0], set()).add(','.join(row[2:]) + '\n')
        fileContents.setdefault(row[1], set()).add(','.join(row[2:]) + '\n')

    timestamp = '_' + '{:%y%m%d%H%M%S}'.format(datetime.datetime.now()) + '_'
    for key in fileContents:
        lines = list(fileContents[key])
        lines.sort()
        with open(str(key) + timestamp + SequenceNum(key) + '.csv', 'w') as f:
            f.writelines(lines)

if __name__ == '__main__':
    filegenerate('test.csv')

此代码使用您提供的test.csv 生成以下文件：

2002_200418103658_000000001.csv

1,1,1,1,1,1,1,1
2,2,2,2,2,2,2,2
4,4,4,4,4,4,4,4

2402_200418103658_000000001.csv

1,1,1,1,1,1,1,1
3,3,3,3,3,3,3,3
4,4,4,4,4,4,4,4
5,5,5,5,5,5,5,5

3333_200418103658_000000001.csv

5,5,5,5,5,5,5,5
6,6,6,6,6,6,6,6