将文件记录到数据框中答案

【问题标题】：log file into a dataframe将文件记录到数据框中
【发布时间】：2020-04-22 03:21:01
【问题描述】：

我有一个日志文件：

20-04-21 15:04:54.355 -07 000000105 INF: Y Motor: Loading Y Motion Profile for 0.830 mm

20-04-21 15:04:54.355 -07 000000105 INF: Y Motor: The Y Motion Profile for 0.830 mm is already loaded, skipping

20-04-21 15:04:54.355 -07 000000105 INF: SipperMotor: Set Sipper Position Down

20-04-21 15:04:54.355 -07 000000105 INF: FPGA: Xmit SIPDOWN\n [SipperMotor]

20-04-21 15:05:07.665 -07 000000136 INF: FPGA: Recv SIPDOWN

20-04-21 15:05:07.665 -07 000000105 INF: FPGA: 'SIPDOWN' command took 0 ms to send, 13300 ms to get response, and 13305 ms overall

没有标题
我想将第一列拆分为日期##-##-##
第二列应该是##:##:##.###
第三列应该是-##
第四列应为XXX：
第五列应将其他所有内容作为文本

【问题讨论】：

标签： python pandas file logging

【解决方案1】：

我会将文件作为单列读取并提取：

df = pd.read_csv('file.txt', header=None, sep='\t')
df = df[0].str.extract('(\S+) (\S+) (\S+) (\S+) (.*)$')

输出：

    0         1               2          3  4
--  --------  ------------  ---  ---------  ----------------------------------------------------------------------------------------------
 0  20-04-21  15:04:54.355  -07  000000105  INF: Y Motor: Loading Y Motion Profile for 0.830 mm
 1  20-04-21  15:04:54.355  -07  000000105  INF: Y Motor: The Y Motion Profile for 0.830 mm is already loaded, skipping
 2  20-04-21  15:04:54.355  -07  000000105  INF: SipperMotor: Set Sipper Position Down
 3  20-04-21  15:04:54.355  -07  000000105  INF: FPGA: Xmit SIPDOWN\n [SipperMotor]
 4  20-04-21  15:05:07.665  -07  000000136  INF: FPGA: Recv SIPDOWN
 5  20-04-21  15:05:07.665  -07  000000105  INF: FPGA: 'SIPDOWN' command took 0 ms to send, 13300 ms to get response, and 13305 ms overall

【讨论】：

【解决方案2】：

因为您有固定宽度的格式化线条。你可以使用pd.read_fwf

df = pd.read_fwf('d1.csv', colspecs=[(0,8),(9,21),(22,25),(26,35),(36,-1)], header=None)

df
           0               1       2        3                                                   4
0   20-04-21    15:04:54.355    -7.0    105.0   INF: Y Motor: Loading Y Motion Profile for 0.8...
1   20-04-21    15:04:54.355    -7.0    105.0   INF: Y Motor: The Y Motion Profile for 0.830 m...
2   20-04-21    15:04:54.355    -7.0    105.0   INF: SipperMotor: Set Sipper Position Down
3   20-04-21    15:04:54.355    -7.0    105.0   INF: FPGA: Xmit SIPDOWN\n [SipperMotor]
4   20-04-21    15:05:07.665    -7.0    136.0   INF: FPGA: Recv SIPDOWN
5   20-04-21    15:05:07.665    -7.0    105.0   INF: FPGA: 'SIPDOWN' command took 0 ms to send...

colspecs : 元组列表 (int, int) 或“推断”。可选

一个元组列表，将每行的固定宽度字段的范围作为半开间隔（即[from, to[）。字符串值‘infer’ 可用于指示解析器尝试从数据的前 100 行中检测未通过skirows 跳过的列规范 (default=’infer’). - doc

【讨论】：