【发布时间】:2022-09-29 20:20:16
【问题描述】:
我只是从网页上抓取链接。我需要循环所有 URL,因为我希望在单个索引上有一个 URL,但是在我将 URLS 放入列表格式并尝试获取单个 URL 而不是完整 URL 之后,我得到列表中每个 URL 的单个字母。这是代码:
import pandas as pd
from bs4 import BeautifulSoup
import requests
from csv import writer
url = \'https://www.pwcs.edu/index\'
req = requests.get(url)
soup = BeautifulSoup(req.text, \'lxml\')
lists = soup.find(\'div\', class_=\'school-dropdown-col\')
for link in lists.find_all(\'a\', href=True):
x = list([link[\'href\']])
print(x[0][3])
标签: python web-scraping