【发布时间】:2021-05-22 16:01:38
【问题描述】:
下面是我想抓取网站并将值存储在 excel 中的代码-
**
import pandas as pd
import requests
from bs4 import BeautifulSoup
from openpyxl import workbook
Name = []
Mob = []
Add = []
E_mail = []
website = []
headers = {
'User-Agent': 'Mozilla/5.0 (Macintosh; Intel Mac OS X 10.12; rv:55.0) Gecko/20100101 Firefox/55.0',
}
url = requests.get("www.example.com", headers=headers)
soup = BeautifulSoup(url.content, 'html.parser')
travel_name = soup.findAll(attrs={'class': 'list-group-item'})
for name in travel_name:
for a in name.findAll('a', attrs={"class": "text-warning"}):
user = a.text
Name.append(user)
pList = name.findAll('p', attrs={"class": "mb-2 text-truncate"})
for p in pList:
# print(p.text)
if p.text.find("Contact:") != -1:
contact = str.replace(p.text, "Contact:", "")
Mob.append(contact)
# print(contact)
if p.text.find("Location:") != -1:
location = str.replace(p.text, "Location:", "")
Add.append(location)
# print(location)
if p.text.find("Email:") != -1:
email = str.replace(p.text, "Email:", "")
E_mail.append(email)
# print(email)
if p.text.find("Website:") != -1:
web = str.replace(p.text, "Website:", "")
website.append(web)
**
我想在 excel 中按列存储值。我试过 df = pd.DataFrame() 但我失败了 [名称、成员、地址、电子邮件、网站]
【问题讨论】:
标签: python python-2.7 python-requests screen-scraping