用beautifulsoup 抓取选择href

Question

我想用 python/BeautifulSoup 抓取一个网站，包括这篇文章：
https://www.electrive.com/2022/02/20/byd-planning-model-3-like-800-volt-sedan-called-seal/

在每篇文章的末尾，您总能找到来源。在上面的链接的情况下，这是：

在本网站上的某些文章中，只给出了一个来源，但有时会给出两个或三个不同的来源。所以代码需要考虑这一点。

理想情况下，我想要以下输出格式：“文本（href）”

xchuxing.com (https://xchuxing.com/article/45850)
cnevpost.com (https://cnevpost.com/2022/02/18/byd-seal-set-to-become-new-tesla-model-3-challenger/) 

这是我的第一个代码：

from bs4 import BeautifulSoup
import requests
import csv

URL = 'https://www.electrive.com/2022/02/20/byd-planning-model-3-like-800-volt-sedan-called-seal/'
(response := requests.get(URL)).raise_for_status()
soup = BeautifulSoup(response.text, 'lxml')
article = soup.find()

source = [c for c in article.find('section', class_='content').find_all('a')]
for link in source[3:]:
        link.get('href')
print (link)   

截至目前的输出：

<a href="https://cnevpost.com/2022/02/18/byd-seal-set-to-become-new-tesla-model-3-challenger/" rel="noopener" target="_blank">cnevpost.com</a>
[Finished in 345ms]

Answer 1

我认为来源总是在文章的最后一段，所以请按照以下步骤提取它们：

from bs4 import BeautifulSoup
import requests
import csv

URL = 'https://www.electrive.com/2022/02/20/byd-planning-model-3-like-800-volt-sedan-called-seal/'
(response := requests.get(URL)).raise_for_status()
soup = BeautifulSoup(response.text, 'lxml')


paragraphs = soup.find('section', class_='content').find_all('p')
# the sources in the last paragraph
sources = paragraphs[-1].find_all('a')
# put the sources name and link in a dict
sources_links = []
for source in sources:
    sources_links.append((source.text, source['href']))

for l in sources_links:
    print(l)

# write in csv
with open('electrive_scrape_source.csv', 'w') as csv_file:
    csv_writer = csv.writer(csv_file)
    csv_writer.writerow(['Source', 'Link'])
    csv_writer.writerows(sources_links)

将数据保存到 csv 文件