如何使用 Beautiful Soup 提取深度嵌套的 <p> 标签

Question

我有以下内容，我正在尝试了解如何使用 Beautiful Soup 提取 <p> 标签副本（我对其他方法持开放态度）。如您所见，<p> 标签并非都嵌套在同一个<div> 中。我用下面的method 试了一下，但这似乎只有在两个<p> 标签都在同一个容器中时才有效。

<div class="top-panel">
  <div class="inside-panel-0">
    <h1 class="h1-title">Some Title</h1>
  </div>
  <div class="inside-panel-0">
    <div class="inside-panel-1">
      <p> I want to extract this copy</p>
    </div>
    <div class="inside-panel-1">
      <p>I want to extract this copy</p>
    </div>
  </div>
</div>

Answer 1

IIUC 尝试

from bs4 import BeautifulSoup

html = """<div class="top-panel">
  <div class="inside-panel-0">
    <h1 class="h1-title">Some Title</h1>
  </div>
  <div class="inside-panel-0">
    <div class="inside-panel-1">
      <p> I want to extract this copy</p>
    </div>
    <div class="inside-panel-1">
      <p>I want to extract this copy</p>
    </div>
  </div>
</div>"""

soup = BeautifulSoup(html, 'lxml')
# find all the p tags that have a parent class of inside-panel-1
soup.findAll({'p': {'class': 'inside-panel-1'}})

[<p> I want to extract this copy</p>, <p>I want to extract this copy</p>]

如果你只想要文字，那么试试

p_tags = soup.findAll({'p': {'class': 'inside-panel-1'}})
[elm.text for elm in p_tags]  
# -> [' I want to extract this copy', 'I want to extract this copy']

Answer 2

由于p标签在div class="inside-panel-1内部，所以我们可以通过调用find_all方法轻松抓取它们，如下所示：

from bs4 import BeautifulSoup

html = """
<div class="top-panel">        
 <div class="inside-panel-0">  
  <h1 class="h1-title">        
   Some Title
  </h1>
 </div>
 <div class="inside-panel-0">  
  <div class="inside-panel-1"> 
   <p>
    I want to extract this copy
   </p>
  </div>
  <div class="inside-panel-1"> 
   <p>
    I want to extract this copy
   </p>
  </div>
 </div>
</div>

"""

soup = BeautifulSoup(html, 'html.parser')
# print(soup.prettify())

p_tags = soup.select('div.top-panel div[class="inside-panel-1"]')
for p_tag in p_tags:
    print(p_tag.get_text(strip=True))

输出：

I want to extract this copy
I want to extract this copy