【问题标题】:Replace <br> with space in BeautifulSoap output用 BeautifulSoap 输出中的空格替换 <br>
【发布时间】:2019-08-30 14:04:37
【问题描述】:

我正在用 BeautifulSoap 抓取一些链接,但它似乎完全忽略了&lt;br&gt; 标签。

这是我正在抓取的网址的源代码的相关部分:

<h1 class="para-title">A quick brown fox jumps over<br>the lazy dog
<span id="something">&#xe800;</span></h1>

这是我的 BeautifulSoap 代码(仅相关部分),用于获取 h1 标签内的文本:

    soup = BeautifulSoup(page, 'html.parser')
    title_box = soup.find('h1', attrs={'class': 'para-title'})
    title = title_box.text.strip()
    print title

这给出了以下输出:

    A quick brown fox jumps overthe lazy dog

而我期待:

    A quick brown fox jumps over the lazy dog

如何在我的代码中将&lt;br&gt; 替换为space

【问题讨论】:

  • 上面的 sn-p 给出了应有的输出A quick brown fox jumps overthe lazy dog some stuff here
  • 是的,我的错误是 span 不包含任何文本。我已经编辑了问题。

标签: python web-scraping beautifulsoup


【解决方案1】:

.get_text() 与分隔符参数一起使用怎么样?

from bs4 import BeautifulSoup

page = '''<h1 class="para-title">A quick brown fox jumps over<br>the lazy dog
<span>some stuff here</span></h1>'''


soup = BeautifulSoup(page, 'html.parser')
title_box = soup.find('h1', attrs={'class': 'para-title'})
title = title_box.get_text(separator=" ").strip()
print (title)   

输出:

print (title)
A quick brown fox jumps over the lazy dog
 some stuff here

【讨论】:

  • 我很抱歉,因为跨度标签不包含任何文本(问题已编辑),所以这对我很有用。谢谢。
【解决方案2】:

在解析之前在html上使用replace()

from bs4 import BeautifulSoup

html = '''<h1 class="para-title">A quick brown fox jumps over<br>the lazy dog
<span>some stuff here</span></h1>'''

html = html.replace("<br>", " ")
soup = BeautifulSoup(html, 'html.parser')
title_box = soup.find('h1', attrs={'class': 'para-title'})
title = title_box.get_text().strip()
print (title)

输出

A quick brown fox jumps over the lazy dog
some stuff here

编辑

对于下面cmets中提到的部分OP;

html = '''<div class="description">Planet Nine was initially proposed to explain the clustering of orbits
Of Planet Nine's other effects, one was unexpected, the perpendicular orbits, and the other two were found after further analysis. Although other mechanisms have been offered for many of these peculiarities, the gravitational influence of Planet Nine is the only one that explains all four.
</div>'''

from bs4 import BeautifulSoup

html = html.replace("\n", ". ")
soup = BeautifulSoup(html, 'html.parser')
div_box = soup.find('div', attrs={'class': 'description'})
divText= div_box.get_text().strip()
print (divText)

输出

Planet Nine was initially proposed to explain the clustering of orbits. Of Planet Nine's other effects, one was unexpected, the perpendicular orbits, and the other two were found after further analysis. Although other mechanisms have been offered for many of these peculiarities, the gravitational influence of Planet Nine is the only one that explains all four..

【讨论】:

  • 在我的代码的不同部分,我正在抓取的文本中有换行符(没有 br,只是换行符)。如何用句点和空格替换换行符?
  • @mumer91 你能发个样本吗?
  • 这是 HTML 示例和我的代码:pastebin.com/Q8AnKvJy P.S.我每 90 分钟只能发布一个问题,所以使用 pastebin。 ;)
  • 谢谢,但我收到错误 TypeError: 'NoneType' object is not callable.
  • @mumer91 你复制粘贴了我发布的代码吗?它经过测试并且工作正常。你在哪一行得到错误?
【解决方案3】:

使用str.replace函数:
print title.replace("&lt;br&gt;", " ")

【讨论】:

  • title 上使用replace("&lt;br&gt;", " ") 将不起作用。在将其传递给 BeautifulSoup 之前,您必须在原始 HTML 上使用它。
猜你喜欢
  • 1970-01-01
  • 2014-10-01
  • 2013-06-19
  • 2016-10-15
  • 2023-03-04
  • 1970-01-01
  • 1970-01-01
  • 2020-10-31
  • 1970-01-01
相关资源
最近更新 更多