无法抓取网站标题 - Python Bs4答案

【问题标题】：Not Able To Scrape Website Title - Python Bs4无法抓取网站标题 - Python Bs4
【发布时间】：2021-11-30 07:12:28
【问题描述】：

我正在尝试获取游戏的标题，但标题我也得到了跨度文本

这是我的代码

import time
import requests,pandas
from bs4 import BeautifulSoup


r = requests.get("https://www.pocketgamer.com/android/best-horror-games/?page=1", headers=        
{'User-agent': 'Mozilla/5.0 (X11; Ubuntu; Linux x86_64; rv:61.0) Gecko/20100101 
Firefox/61.0'})
c = r.content
bs4 = BeautifulSoup(c,"html.parser")

all = bs4.find_all("h3",{"class":"indent"}) 
print(all)

输出

[<h3 class="indent">
<div><span>1</span></div>
Fran Bow </h3>, <h3 class="indent">
<div><span>2</span></div>
Bendy and the Ink Machine </h3>, <h3 class="indent">
<div><span>3</span></div>
Five Nights at Freddy's </h3>, <h3 class="indent">
<div><span>4</span></div>
Sanitarium </h3>, <h3 class="indent">
<div><span>5</span></div>
OXENFREE </h3>, <h3 class="indent">
<div><span>6</span></div>
Thimbleweed Park </h3>, <h3 class="indent">
<div><span>7</span></div>
Samsara Room </h3>, <h3 class="indent">

我也试过这段代码，但没有用

#all = all.find_all("h3")[0].text

【问题讨论】：

标签： web-scraping beautifulsoup

【解决方案1】：

如何解决？

因为您想要获取的文本始终是<h3> 中的最后一个元素，您可以通过contents 的<h3> 提取它。

element.contents[-1]

要获取文本迭代结果集：

for x in bs4.find_all("h3",{"class":"indent"}):
    print(x.contents[-1].get_text(strip=True))

示例

import requests,pandas
from bs4 import BeautifulSoup


r = requests.get("https://www.pocketgamer.com/android/best-horror-games/?page=1", 
headers={'User-agent': 'Mozilla/5.0 (X11; Ubuntu; Linux x86_64; rv:61.0) Gecko/20100101 Firefox/61.0'})
c = r.content
bs4 = BeautifulSoup(c,"html.parser")

all = [x.contents[-1].get_text(strip=True) for x in bs4.find_all("h3",{"class":"indent"})]
print(all)

输出

['Fran Bow', 'Bendy and the Ink Machine', "Five Nights at Freddy's", 'Sanitarium', 'OXENFREE', 'Thimbleweed Park', 'Samsara Room', 'Into the Dead 2', 'Slayaway Camp', 'Eyes - the horror game', 'Slendrina:The Cellar', 'Hello Neighbor', 'Alien: Blackout', 'Rest in Pieces', 'Friday the 13th: Killer Puzzle', 'I Am Innocent', 'Detention', 'Limbo', 'Knock-Knock', 'Sara Is Missing', 'Death Park: Scary Horror Clown', 'Horror Hospital 2', 'Horrorfield - Multiplayer Survival Horror Game', 'Erich Sann: Horror in the scary Academy', 'The Innsmouth Case']

【讨论】：