如何通过使用 BeautifulSoup 抓取 IP 地址并输出到 CSV？

Question

import requests    
from bs4 import BeautifulSoup

url ='https://myip.ms/browse/blacklist/Blacklist_IP_Blacklist_IP_Addresses_Live_Database_Real-time'

response = requests.get(url)

data = response.text

soup = BeautifulSoup(data, 'html.parser')

ipList = soup.find("td",{"class": "row_name"})

rows = ipList.findAll('td')
for tr in rows:
  cols = td.findAll('td')
  if len(cols) > 0:
     print (ip.cols.text.strip())

我正在使用 BeautifulSoup 进行网页抓取，但遇到了一些问题。我可以知道为什么我无法从数据库表中抓取 IP 地址。如何将结果输出到 CSV 文件？

Answer 1

问题是你使用find()作为ipList，它只获取一个ip，你可以使用findall()或select这将返回ip数组。

import requests
from bs4 import BeautifulSoup
url ='https://myip.ms/browse/blacklist/Blacklist_IP_Blacklist_IP_Addresses_Live_Database_Real-time'
response = requests.get(url).content
soup = BeautifulSoup(response, 'html.parser')
ipList = soup.select(".row_name")
with open('ip_output.csv', 'w') as f:
    for ips in ipList:
        f.write(ips.find('a').text + '\n')

csv 中的输出

195.154.251.86
37.140.192.194
80.94.174.55
175.103.39.28
90.173.129.250
51.15.146.121
...
...