合并多个csv文件[关闭]

Question

我有 3 个 csv 文件，我想如何将这 3 个文件写入单个 csv 文件。 例如

file1.csv

a b c d
1 2 3 4 
5 6 7 8

文件 2.csv

e f g h
13 14 15 16
17 18 19 20

file3.csv

i j k l 
9 10 11 12 
21 22 23 24

所需的输出如下

  a b c d e   f g  h  i j  k  l
  1 2 3 4 13 14 15 16 9 10 11 12
  5 6 7 8 17 18 19 20 21 22 23 24

Answer 1

这些家伙是对的，你不应该要求代码。尽管如此，我发现这项任务足以吸引我花三分钟时间来解决这个问题：

import csv

allColumns = []
for dataFileName in [ 'a.csv', 'b.csv', 'c.csv' ]:
  with open(dataFileName) as dataFile:
    fileColumns = zip(*list(csv.reader(dataFile, delimiter=' ')))
    allColumns += fileColumns

allRows = zip(*allColumns)

with open('combined.csv', 'w') as resultFile:
  writer = csv.writer(resultFile, delimiter=' ')
  for row in allRows:
    writer.writerow(row)

请注意，此解决方案可能不适用于大输入。它还假设所有文件都有相同数量的行（行），如果不是这种情况可能会中断。

Answer 2

一个想法可能是使用 zip 功能

file1 = "a b c d\n1 2 3 4\n5 6 7 8"
file2 = "e f g h\n13 14 15 16\n17 18 19 20"
file3 = "i j k l\n9 10 11 12\n21 22 23 24"

merged_file =[i+" " +j+" " +k for i,j,k in zip(file1.split('\n'),file2.split('\n'),file3.split('\n'))]
for i in merged_file:
   print i

Answer 3

考虑到所有文件都有相同的行。此解决方案也适用于大型输入，因为一次仅将 3 行（每个文件中的一行）带入内存。

import csv
with open('foo1.txt') as f1, open('foo2.txt') as f2, \
     open('foo2.txt') as f3, open('out.txt', 'w') as f_out:

     writer = csv.writer(f_out, delimiter=' ')
     readers = [csv.reader(x, delimiter=' ') for x in (f1, f2, f3)]
     while True:
         try:
             writer.writerow([y for w in readers for y in next(w)])
         except StopIteration:
             break

上述代码的基于 for 循环的版本，但这需要首先迭代其中一个文件以获得行数：

import csv
with open('foo1.txt') as f1, open('foo2.txt') as f2, \
     open('foo2.txt') as f3, open('out.txt', 'w') as f_out:

     writer = csv.writer(f_out, delimiter=' ')
     lines = sum(1 for _ in f1) #Number of lines in f1
     f1.seek(0)                 #Move the file pointer to the start of file 
     readers = [csv.reader(x, delimiter=' ') for x in (f1, f2, f3)]
     for _ in range(lines):
         writer.writerow([y for w in readers for y in next(w)])

Answer 4

Python Pandas 方式。

（上述代码的略微改进版本）

import pandas as pd

files = ['file1.csv', 'file2.csv', 'file3.csv']

df_combined = pd.concat(map(pd.read_csv, files))
df_combined.to_csv('output.csv', index=None)

然后你得到组合的csv文件output.csv

Unix 命令行方式。

paste -d" " file1.txt file2.txt

如果您使用的是 UNIX 类型的操作系统，请检查您是否只关心合并文件how to merge two files consistently line by line

神速。

Answer 5

您可以使用pandas，这是一种数据操作工具。

import pandas as pd

df1 = pd.read_csv('file1.csv')
df2 = pd.read_csv('file2.csv')
df3 = pd.read_csv('file3.csv')

df_combined = pd.concat([df1, df2, df3],axis=1)
df_combined.to_csv('output.csv', index=None)

然后你得到组合的csv文件output.csv

Answer 6

inputs = 'file1.csv', 'file2.csv', 'file3.csv'

with open('out.csv','w') as output:
    for line in zip(*map(open, inputs)):
        output.write('%s\n'%' '.join(i.strip() for i in line))

编辑：
这是一个详细的版本。

inputs = 'file1.csv', 'file2.csv', 'file3.csv'

# open all input files
inputs = map(open, inputs)

with open('out.csv','w') as output:

    # iter over all the input files at the same time
    for line in zip(*inputs):

        # format the output line from input lines
        line = ' '.join(i.strip() for i in line)

        output.write('%s\n' % line)

Answer 7

除了第一个答案，这是正确的答案，您可以通过以下方式使其更通用地处理文件夹中的任意数量的 csv 文件：

import os
import pandas as pd

folder = r"C:\MyFolder"

frames = [pd.read_csv(os.path.join(folder,name) for name in os.listdir(folder) if name.endswith('.csv')]

merged = pd.concat(frames)

文档： http://pandas.pydata.org/pandas-docs/dev/merging.html

Answer 8

首先想到的是使用 pandas 模块，就像 waitingkuo 的回答一样。但我想你也可以使用 DictWriter ......

import csv

# Initialize output file
header = [x for x in 'abcdefghijkl']    
output = csv.DictWriter(open('final_output.csv', 'wb'), fieldnames = header)
output.writerow(dict(zip(header, header))) 

# Compile contents of all three files into a single dictionary, outputdict
outputdict = {key:[] for key in header}
for fname in ['file1.csv', 'file2.csv', 'file3.csv']: 
    f = csv.DictReader(open(fname, 'r')) 
   [(outputdict[k]).append(line[k]) for k in line for line in f]


# Transfer the contents of outputdict into a csv file
[output.writerow(l) for l in outputdict]