从日志写入csv时字符串替换

Question

我正在尝试使用 python (mainly this example) 将日志文件转换为 csv

import csv

with open('/home/user/Downloads/my.log') as file:
    lines = file.read().splitlines()
    lines = [lines[x:x+2] for x in range(0, len(lines), 3)]

    with open('test.csv', 'w+') as csvfile:
        w = csv.writer(csvfile)
        w.writerows(lines)
        print "done"

我的原始日志文件如下所示：

2017-08-09 -> 11:30:01
Temp=29.0*  Humidity=30.0%

2017-08-09 -> 11:40:01
Temp=29.0*  Humidity=33.0%

从上面的示例代码，我可以把它转换成这种格式

2017-08-08 -> 08:50:01,Temp=33.0*  Humidity=38.0%
2017-08-08 -> 09:00:01,Temp=37.0*  Humidity=40.0%

但我需要我的最终 csv 看起来像这样

2017-08-08,08:50:01,33.0*,38.0%, 
2017-08-08,09:00:01,37.0*,40.0%

我用过lines = lines.replace("->",",")，我得到了

AttributeError: 'list' 对象没有属性 'replace'

据我了解，python 不能替换它在内存中处理的文本文件。那我能做什么？我可以用什么方法来净化最终的文本？

我的python知识不高级，还在学习中。所以如果有错误或遗漏的步骤，请纠正我。

提前致谢

Answer 1

您使用replace 作为列表方法，因为lines 是一个包含字符串的列表，如果您想对其进行转换，请使用map 或list comprehension：

lines = [i.replace("->",",") for i in lines]

或

lines = map(lambda x: x.replace("->",","), lines)

您可以链接转换以获得最终结果：

lines = map(lambda x: x.replace("->",",").replace("Temp=", "").replace("  Humidity=", ",").replace(" ", ""), 
            lines)

你有一个live example

Answer 2

更新：完整修改代码

with open('/home/user/Downloads/my.log') as file, open("output.csv", "w") as outfile:
    lines = file.readlines()
    lines = [','.join([i.strip() for i in lines[x:x+2]]) for x in range(0, len(lines), 3)]
    lines = [i.replace("->",",").replace(" ","").replace("Temp=","").replace("Humidity=",",") for i in lines]

    outfile.write("Date,Time,Temp,Humidity")

    for line in lines:
        outfile.write(line)

输出

Date,Time,Temp,Humidity
2017-08-08,08:50:01,33.0*,38.0%
2017-08-08,09:00:01,37.0*,40.0%

---

工作示例

datafile="""2017-08-09 -> 11:30:01
Temp=29.0*  Humidity=30.0%

2017-08-09 -> 11:40:01
Temp=29.0*  Humidity=33.0%"""

lines = io.StringIO(datafile).readlines()
lines = [','.join([i.strip() for i in lines[x:x+2]]) for x in range(0, len(lines), 3)]
lines = [i.replace("->",",").replace(" ","").replace("Temp=","").replace("Humidity=",",") for i in lines]
lines

返回：

['2017-08-09,11:30:01,29.0*,30.0%', '2017-08-09,11:40:01,29.0*,33.0%']

Answer 3

replace() 方法可以在字符串上调用，而不是在列表上，所以lines.replace("->",",") 不正确。您可以对文件中的整个文本进行操作，然后将其分成几行：

with open('/home/user/Downloads/my.log') as file:
    text = file.read().replace("->", ",")
    lines = text.splitlines()
    lines = [lines[x:x+2] for x in range(0, len(lines), 3)]

Answer 4

您愿意使用正则表达式吗？如果是这样，请使用以下模式从每个字符串中提取组 1、2、3 和 4：

(\d{4}-\d{2}-\d{2}) -> (\d{2}:\d{2}:\d{2}),Temp=([0-9.]+)\*\s+Humidity=([0-9.]+)

(see it working here)

您将使用 re 库。

import csv
import re

with open('/home/user/Downloads/my.log') as file:
    lines = file.read().splitlines()
    lines = [lines[x:x+2] for x in range(0, len(lines), 3)]

    pattern = '(\d{4}-\d{2}-\d{2}) -> (\d{2}:\d{2}:\d{2}),Temp=([0-9.]+)\*\s+Humidity=([0-9.]+)'

    with open("output.csv", "w") as f:
        f.write("Date,Time,Temp,Humidity\n")
        print("Date,Time,Temp,Humidity")
        for line in lines:
            m = re.search(pattern, line)
            f.write("{}, {}, {}, {}\n".format(m.group(1), m.group(2), m.group(3), m.group(4)))
            print("{}, {}, {}, {}".format(m.group(1), m.group(2), m.group(3), m.group(4)))

Answer 5

这会创建一个字符串列表：

lines = file.read().splitlines()

然后这会创建一个字符串列表：

lines = [lines[x:x+2] for x in range(0, len(lines), 3)]

替换作用于字符串，而不是列表。有很多方法可以解决这个问题：

# 1. do replace on original string, before splitting.
lines = file.read().replace("->", ",") 

# 2. or do replace on elements of list, before creating list of lists
lines = file.read().splitlines()
lines = [i.replace("->",",") for i in lines]

# 3. or replace on each element in list of lists 
# (not implemented)