解决方案
您需要使用以下正则表达式模式以及pandas.Series.str.findall() 才能快速轻松地获取它。
我还做了一个便利功能:process_logdata(),这样你就可以直接使用了。此答案的底部提供了便利功能。
df = process_logdata(log_file_name='logfile.txt')
print(df)
逻辑:
这里是便利函数的逻辑,process_logdata()。
# regex pattern
pattern = '\s*(\w{3}\s+\d{1,2}\s+\d{2}:\d{2}:\d{2})\s+(\S+)\s+(\S+?:)\s+(.*)'
# read log file
df = pd.read_csv('logfile.txt', header=None).rename(columns={0: 'logline'})
# process data
ds = df.logline.str.strip().str.findall(pattern)
a = np.array([list(e) for e in ds]).reshape(ds.size,-1)
# finalize processed data as a dataframe
df = pd.DataFrame(a, columns=['Timestamp', 'A1', 'A3', 'A3'])
print(df)
示例
这里我们使用虚拟数据(以字符串形式提供)。首先,我们将其加载到 pandas 数据帧中,然后对其进行处理。
import numpy as np
import pandas as pd
import re
from io import StringIO
s = """
Jan 9 01:04:49 syntax sshd(pam_unix)[21354]: authentication failure; logname= uid=0 euid=0 tty=NODEVssh ruser= rhost=120-123-141-4.hinet-ip.hinet.com
Jul 10 04:17:11 syntax su(pam_unix)[95367]: session opened for user abc by (uid=0)
May 1 14:06:19 syntax su(pam_unix)[95367]: session closed for user abc
Oct 23 18:08:26 syntax logrotate: ALERT exited abnormally with [1]
Jun 14 21:42:52 syntax su(pam_unix)[95367]: session opened for user cbx by (uid=0)
"""
s = re.sub('\n\s*\n', '\n', s).strip()
#print(s)
df = pd.read_csv(StringIO(s), header=None).rename(columns={0: 'logline'})
pattern = '\s*(\w{3}\s+\d{1,2}\s+\d{2}:\d{2}:\d{2})\s+(\S+)\s+(\S+?:)\s+(.*)'
ds = df.logline.str.strip().str.findall(pattern)
a = np.array([list(e) for e in ds]).reshape(ds.size,-1)
df = pd.DataFrame(a, columns=['Timestamp', 'A1', 'A3', 'A3'])
print(df)
输出:
便利功能
import numpy as np
import pandas as pd
import re
def process_logdata(log_file_name):
"""Returns a dataframe created from the log file.
"""
## Define regex pattern
pattern = '\s*(\w{3}\s+\d{1,2}\s+\d{2}:\d{2}:\d{2})\s+(\S+)\s+(\S+?:)\s+(.*)'
## Read log file
df = (pd
.read_csv(log_file_name, header=None)
.rename(columns={0: 'logline'})
)
## Process data
ds = df['logline']str.strip().str.findall(pattern)
a = np.array([list(e) for e in ds]).reshape(ds.size,-1)
## Finalize processed data as a dataframe
df = pd.DataFrame(a, columns=['Timestamp', 'A1', 'A3', 'A3'])
return df