我可以在 soup.find_all() 函数中搜索多个 HTML 元素吗？

Question

我正在尝试从网站上抓取浏览次数最多的头条新闻。我想要的文本的类选择器与页面上的其他项目共享常用词。例如，我想要标签和类“black_color”之间的文本。其他项目使用该标签并具有“color_black hover_color_gray_90”类，我不希望包括这些。我在想我可以使用更多的 HTML 元素来更具体，但我不确定如何合并它们。

from bs4 import BeautifulSoup

def getHeadlines():
    url = "https://www.bostonglobe.com/"

    source_code = requests.get(url)

    plainText = source_code.text

    soup = BeautifulSoup(plainText, "html.parser")

    #results = soup.find_all("h2",{"class":"headline"})
    results = soup.find_all("a",{"class":"black_color"})
    

    with open("headlines.txt", "w", encoding="utf-8") as f:
        for i in results:
            f.write(str(i.text + ' \n' + '\n'))
        

getHeadlines()

Answer 1

我认为查看<a> 标签实际上可能比使用匹配的<h2> 更难，它有一个'headline' 类。

试试这个：

soup = BeautifulSoup(source_code.text, "html.parser")

for headline in soup.find_all("h2", class_="headline"):
    print(headline.text)

输出：

Boston College outbreak worries epidemiologists, students, community
Nantucket finds ‘community spread’ of COVID-19 among tradespeople The town’s Select Board and Board of Health will hold an emergency meeting at 10 a.m. Monday to consider placing restrictions on some of those trades, officials said Friday. 
Weddings in a pandemic: Welcome to the anxiety vortexNewlyweds are spending their honeymoons praying they don’t hear from COVID‐19 contact tracers. Relatives are agonizing over “damned if we RSVP yes, damned if we RSVP no” decisions. Wedding planners are adding contract clauses specifying they’ll walk off the job if social distancing rules are violated. 
Fauci says US should plan to ‘hunker down’ for fall and winter
Of struggling area, Walsh says, ‘We have to get it better under control’ 
...

看了一会，我觉得<a>这个标签实际上可能有一些类是动态添加的，BS不会接。仅搜索 color_black 类并排除 color_black hover_color_gray_90 类会产生您不想要的标题类（例如“体育”），即使当我查看实际网页源代码时，我发现它在你所指示的方式。

这通常是对页面进行加载后 CSS 更改的好兆头。 （我可能错了，但无论哪种情况，我都希望<h2> 方法能满足您的需求。）