使用 shell 脚本读取、比较和保存 2 个文件（比听起来要多一点）[重复]

Question

所以我之前没有完全得到我想要的答案，所以我们又来了；

我正在创建一个脚本，用于根据预定义的黑 IP 列表从 .csv 日志文件中搜索 IP。

它首先导入日志文件，然后从中解析IP，然后根据预定义的黑IP列表搜索解析的IP，最后它需要询问用户（如果找到任何结果）将结果保存到原始日志文件那是进口的。

文件 1 是代码中 IP-output.csv 的示例。

文件 2 是代码中 $filename 的示例（原始导入的 .csv）。

文件 1：

107.147.166.60 ,SUSPICIOUS IP
107.147.167.26 ,SUSPICIOUS IP
108.48.185.186 ,SUSPICIOUS IP
108.51.114.130 ,SUSPICIOUS IP
142.255.102.68 ,SUSPICIOUS IP

文件 2：

outlook.office365.com ,174.203.0.118 ,UserLoginFailed
outlook.office365.com ,107.147.166.60 ,UserLoginFailed
outlook.office365.com ,107.147.167.26 ,UserLoginFailed
outlook.office365.com ,174.205.17.24 ,UserLoginFailed
outlook.office365.com ,108.48.185.186 ,UserLoginFailed
outlook.office365.com ,174.226.15.21 ,UserLoginFailed
outlook.office365.com ,108.51.114.130 ,UserLoginFailed
outlook.office365.com ,67.180.23.93 ,UserLoginFailed
outlook.office365.com ,142.255.102.68 ,UserLoginFailed
outlook.office365.com ,164.106.75.235 ,UserLoginFailed

我想把文件 2 改成这样：

outlook.office365.com ,174.203.0.118 ,UserLoginFailed
outlook.office365.com ,107.147.166.60 ,UserLoginFailed ,SUSPICIOUS IP
outlook.office365.com ,107.147.167.26 ,UserLoginFailed ,SUSPICIOUS IP
outlook.office365.com ,174.205.17.24 ,UserLoginFailed
outlook.office365.com ,108.48.185.186 ,UserLoginFailed ,SUSPICIOUS IP
outlook.office365.com ,174.226.15.21 ,UserLoginFailed
outlook.office365.com ,108.51.114.130 ,UserLoginFailed ,SUSPICIOUS IP
outlook.office365.com ,67.180.23.93 ,UserLoginFailed
outlook.office365.com ,142.255.102.68 ,UserLoginFailed ,SUSPICIOUS IP
outlook.office365.com ,164.106.75.235 ,UserLoginFailed

这是我创建的脚本：

#!/bin/bash
#
# IP Blacklist Checker
#Import .csv (File within working directory)
echo "Please import a .csv log file to parse/search the IP(s) and UserAgents: "
read filename
#Parsing IPs from .csv log file
echo "Parsing IP(s) from imported log file..."
grep -Eo '([0-9]{1,3}\.){3}[0-9]{1,3}' $filename | sort | uniq > IP-list.txt
echo 'Done'
awk 'END {print NR,"IP(s) Found in imported log file"}' IP-list.txt
echo 'IPs found in imported log file:'
cat IP-list.txt
#searches parsed ip's against blacked ip lists
echo 'Searching parsed IP(s) from pre-defined Blacked IP List Databases...'
fgrep -w -f "IP-list.txt" "IPlist.txt" > IP-output.txt
awk 'END {print NR,"IP(s) Found Blacked IP List Databases"}' IP-output.txt
echo 'Suspicious IPs found in Blacked IP List Databases:'
cat IP-output.txt
while true; do
read -p "Do you want to add results to log file?" yn
case $yn in
    [Yy]* ) grep -Ff IP-output.txt $filename | sed 's/$/ ,SUSPICIOUS IP/' > IP-output.csv && awk 'FNR==NR {m[$1]=$0; next} {for (i in m) {match($0,i); val=substr($0, RSTART, RLENGTH); if (val) {sub(val, m[i]); print; next}};} 1' IP-output.csv $filename > $filename; break;;
    [Nn]* ) break;;
    * ) echo "Please answer yes or no.";;
esac
done
echo "Finished searching parsed IP(s) from pre-defined Blacked IP List Databases."
rm IP-list.txt IP-output.csv IP-output.txt 

我正在导入的日志文件非常长，有 15-20 列，而 IPlist.txt（涂黑的 IP）中有超过 15000 个 IP。将结果保存到同一个日志文件后，.csv 文件为空，如果我以不同的名称保存它，所有列都会乱序，并且 IP 列旁边出现“，可疑 IP”列，我需要它而不是在最后一列（行尾）。

我也不知道如何仅在找到任何内容时提示保存文件，如果不仅提示未找到！

我得到的结果：

 outlook.office365.com ,174.203.0.118 ,UserLoginFailed
 outlook.office365.com ,107.147.166.60 ,SUSPICIOUS IP ,UserLoginFailed
 outlook.office365.com ,107.147.167.26 ,SUSPICIOUS IP ,UserLoginFailed
 outlook.office365.com ,174.205.17.24 ,UserLoginFailed
 outlook.office365.com ,108.48.185.186 ,SUSPICIOUS IP ,UserLoginFailed
 outlook.office365.com ,174.226.15.21 ,UserLoginFailed
 outlook.office365.com ,108.51.114.130 ,SUSPICIOUS IP ,UserLoginFailed
 outlook.office365.com ,67.180.23.93 ,UserLoginFailed
 outlook.office365.com ,142.255.102.68 ,SUSPICIOUS IP ,UserLoginFailed
 outlook.office365.com ,164.106.75.235 ,UserLoginFailed

Answer 1

用于处理文本的通用标准 UNIX 工具是 awk：

$ awk '
    BEGIN { FS=OFS=" ," }
    NR==FNR { a[$1]=$2; next }
    { print $0 ($2 in a ? OFS a[$2] : "") }
' file1 file2
outlook.office365.com ,174.203.0.118 ,UserLoginFailed
outlook.office365.com ,107.147.166.60 ,UserLoginFailed ,SUSPICIOUS IP
outlook.office365.com ,107.147.167.26 ,UserLoginFailed ,SUSPICIOUS IP
outlook.office365.com ,174.205.17.24 ,UserLoginFailed
outlook.office365.com ,108.48.185.186 ,UserLoginFailed ,SUSPICIOUS IP
outlook.office365.com ,174.226.15.21 ,UserLoginFailed
outlook.office365.com ,108.51.114.130 ,UserLoginFailed ,SUSPICIOUS IP
outlook.office365.com ,67.180.23.93 ,UserLoginFailed
outlook.office365.com ,142.255.102.68 ,UserLoginFailed ,SUSPICIOUS IP
outlook.office365.com ,164.106.75.235 ,UserLoginFailed