在同一写入操作中替换多个 IP 地址答案

【问题标题】：Replace multiple IP address in same write operation在同一写入操作中替换多个 IP 地址
【发布时间】：2018-09-25 02:02:21
【问题描述】：

我正在尝试解析日志文件并替换其中的所有 IP。这对我来说很简单，但我想做的是通过跟踪它是哪个 IP 地址来替换 IP，例如将其视为我的日志文件：

abcdef 192.168.1.1
kbckdbc 10.10.10.10
abcdef 192.168.1.1
yuosdj 100.100.100.100

我希望看到输出为：

abcdef IP_1
kbckdbc IP_2
abcdef IP_1
yuosdj IP_3

我怎样才能做到这一点？

这是我目前所拥有的：

ip_list = []
_IP_RE = re.compile(r"[0-9]{1,3}\.[0-9]{1,3}\.[0-9]{1,3}\.[0-9]{1,3}", re.S)
counter = 0
f1 = open('logfile.txt', 'r')


for line in f1:
    for matchedip in re.findall(r"[0-9]{1,3}\.[0-9]{1,3}\.[0-9]{1,3}\.[0-9]{1,3}",line):
        if matchedip in ip_list:
            matchedip =  '<IP_Address_'+str(ip_list.index(matchedip)+1)+'>'        
        else:
            counter = counter + 1       
            ip_list.append(matchedip)
            matchedip = '<IP_Address_'+str(counter)+'>'
        print matchedip

这是一个测试文件：

2018-09-13 19:00:00,317 INFO  -util.SSHUtil: Waiting for channel close
2018-09-13 19:00:01,317 INFO  -util.SSHUtil: Waiting for channel close
2018-09-13 19:00:01,891 INFO  -filters.BasicAuthFilter: Client IP:192.168.100.98
2018-09-13 19:00:01,891 INFO  -filters.BasicAuthFilter: Validating token ... 
2018-09-13 19:00:01,892 INFO  -authentication.Tokenization: Token:192.168.100.98:20180913_183401is present in map
2018-09-13 19:00:01,892 INFO  -configure.ConfigStatusCollector: status.
2018-09-13 19:00:01,909 INFO  -filters.BasicAuthFilter: Client IP:192.168.100.98
2018-09-13 19:00:01,909 INFO  -filters.BasicAuthFilter: Validating token ... 
2018-09-13 19:00:01,910 INFO  -authentication.Tokenization: Token:192.168.100.98:20180913_183401is present in map
2018-09-13 19:00:01,910 INFO  -restadapter.ConfigStatusService: configuration status.
2018-09-13 19:00:01,910 INFO  -configure.Collector: Getting configuration status.
2018-09-13 19:00:02,318 INFO  -util.SSHUtil: Processing the ssh command execution results standard output.
2018-09-13 19:00:02,318 INFO  -util.SSHUtil: Processing the ssh command execution standard error.
2018-09-13 19:00:02,318 INFO  -util.SSHUtil: Remote command using SSH execution status: Host     : [10.2.251.129]   User     : [root]   Password : [***********]    Command  : [shell ntpdate -u 132.132.0.88]  STATUS   : [0]
2018-09-13 19:00:02,318 INFO  -util.SSHUtil:    STDOUT   : [Shell access is granted to root
            14 Sep 01:00:01 ntpdate[16063]: adjust time server 132.132.0.88 offset 0.353427 sec
]
2018-09-13 19:00:02,318 INFO  -util.SSHUtil:    STDERR   : []
2018-09-13 19:00:02,318 INFO  -util.SSHUtil: Successfully executed remote command using SSH.
2018-09-13 19:00:02,318 INFO  Successfully executed the command on VCenter :10.2.251.129

【问题讨论】：

不是一个答案，而是一个建议：由于 IPv4 是 32 位数字，因此在每个标识符中实际编码数字本身是否会有问题（如十六进制、base36 等）？例如：192.168.1.1 (0xc0a80101) 将变为 IP_c0a80101 (hex) 或 IP_1hge135 (base36)。不像IP_1、IP_2 等那样易于阅读，但具有确定性和一致性。

标签： python regex list

【解决方案1】：

您可以使用字典：

val = {}
result = ''
count = 1
content = """
abcdef 192.168.1.1
kbckdbc 10.10.10.10
abcdef 192.168.1.1
yuosdj 100.100.100.100
sadfsdf 192.168.1.1
newstuff 342.344.23.2
yuosdj 100.100.100.100
"""

data = [i.split() for i in filter(None, content.split('\n'))]
for a, b in data:
  if b not in val:
    result += f'{a} {count}\n'
    val[b] = count
    count += 1
  else:
    result += f'{a} {val[b]}\n'

print(result)

输出：

abcdef 1
kbckdbc 2
abcdef 1
yuosdj 3
sadfsdf 1
newstuff 4
yuosdj 3

编辑：要更新文件中的 IP，您可以使用 re:

import typing, re
def change_ips(ips:typing.List[str]) -> typing.Generator[str, None, None]:
   val = {}
   count = 1
   for i in ips:
     if i not in val:
       yield f'IP_{count}'
       val[i] = count
       count += 1
     else:
       yield f'IP_{val[i]}'


with open('filename.txt') as f:
  content = f.read()
  with open('filename.txt', 'w') as f1:
    f1.write(re.sub('\d+\.\d+\.\d+\.\d+', '{}', content).format(*change_ips(re.findall('\d+\.\d+\.\d+\.\d+', content))))

输出：

2018-09-13 19:00:00,317 INFO  -util.SSHUtil: Waiting for channel close
2018-09-13 19:00:01,317 INFO  -util.SSHUtil: Waiting for channel close
2018-09-13 19:00:01,891 INFO  -filters.BasicAuthFilter: Client IP:IP_1
2018-09-13 19:00:01,891 INFO  -filters.BasicAuthFilter: Validating token ... 
2018-09-13 19:00:01,892 INFO  -authentication.Tokenization: Token:IP_1:20180913_183401is present in map
2018-09-13 19:00:01,892 INFO  -configure.ConfigStatusCollector: status.
2018-09-13 19:00:01,909 INFO  -filters.BasicAuthFilter: Client IP:IP_1
2018-09-13 19:00:01,909 INFO  -filters.BasicAuthFilter: Validating token ... 
2018-09-13 19:00:01,910 INFO  -authentication.Tokenization: Token:IP_1:20180913_183401is present in map
2018-09-13 19:00:01,910 INFO  -restadapter.ConfigStatusService: configuration status.
2018-09-13 19:00:01,910 INFO  -configure.Collector: Getting configuration status.
2018-09-13 19:00:02,318 INFO  -util.SSHUtil: Processing the ssh command execution results standard output.
2018-09-13 19:00:02,318 INFO  -util.SSHUtil: Processing the ssh command execution standard error.
2018-09-13 19:00:02,318 INFO  -util.SSHUtil: Remote command using SSH execution status: Host     : [IP_2]   User     : [root]   Password : [***********]    Command  : [shell ntpdate -u IP_3]  STATUS   : [0]
2018-09-13 19:00:02,318 INFO  -util.SSHUtil:    STDOUT   : [Shell access is granted to root
        14 Sep 01:00:01 ntpdate[16063]: adjust time server IP_3 offset 0.353427 sec]
2018-09-13 19:00:02,318 INFO  -util.SSHUtil:    STDERR   : []
2018-09-13 19:00:02,318 INFO  -util.SSHUtil: Successfully executedremote command using SSH.
2018-09-13 19:00:02,318 INFO  Successfully executed the command on VCenter :IP_2

【讨论】：

我在哪里使用正则表达式？另外，如果每行有两个或三个 ip，它是否有效？
@PetPan 你不需要正则表达式来解决这个问题。该解决方案将适用于多个不同的 ip。请查看我最近的编辑。
我收回了，这个解决方案对我不起作用。我已经用相关的测试文件更新了原帖
@PetPan 抱歉回复晚了。您希望从新输入中得到什么输出？ INFO后面的数据是否代替了“IP”？
没有，只是 ip 部分需要被屏蔽为，等

【解决方案2】：

这是我的建议。如果每行只有一个 ip，这将起作用。

import re

ip2idx = {}
_IP_RE = re.compile(r"[0-9]{1,3}\.[0-9]{1,3}\.[0-9]{1,3}\.[0-9]{1,3}", re.S)
counter = 1

with open('logfile.txt') as f1:
    for line in f1:
        line = line.rstrip()
        # use the compiled regex
        m = _IP_RE.search(line)
        if m:
            ip = m.group(0)
            idx = ip2idx.get(ip)
            if idx is None:
                ip2idx[ip] = counter
                idx = counter
                counter += 1

            print(line[:m.start()] + 'IP_'+str(idx) + line[m.end():])

【讨论】：