【问题标题】:Derive text between p elements在 p 元素之间派生文本
【发布时间】:2020-02-03 10:16:46
【问题描述】:

我想为一堆下载的文件提取 P 强元素之间的文本。 我想要 P strong “Executives” 和 P strong “Analysts” 之间的所有 P 文本,我附上了一个 html 示例,请参阅example 我知道如何加载 html,但我不知道如何使用 BS4 提取前面提到的数据:

import textwrap
import os
from bs4 import BeautifulSoup

directory ='C:/test/out'
for filename in os.listdir(directory):
    if filename.endswith('.html'):
        fname = os.path.join(directory,filename)
        with open(fname, 'r') as f:
            soup = BeautifulSoup(f.read(),'html.parser')

html 示例:

</header><div id="a-cont"><div class="p p1"></div><div class="sa-art article-width" id="a-body"><p>Apple, Inc. (NASDAQ:<a href="https://seekingalpha.com/symbol/AAPL" title="Apple Inc.">AAPL</a>)</p>
<p>Q4 2016 Earnings Call</p>
<p>October 25, 2016 5:00 pm ET</p>
<p><strong>Executives</strong></p>
<p>Nancy Paxton - Apple, Inc.</p>
<p>Timothy Donald Cook - Apple, Inc.</p>
<p>Luca Maestri - Apple, Inc.</p>
<p><strong>Analysts</strong></p>
<p>Eugene Charles Munster - Piper Jaffray &amp; Co.</p>
<p>Kathryn Lynn Huberty - Morgan Stanley &amp; Co. LLC</p>
<p>Shannon S. Cross - Cross Research LLC</p>
<p>Antonio M. Sacconaghi - Sanford C. Bernstein &amp; Co. LLC</p>
<p>Simona K. Jankowski - Goldman Sachs &amp; Co.</p>
<p>Steven M. Milunovich - UBS Securities LLC</p>
<p>Wamsi Mohan - Bank of America Merrill Lynch</p>
<p>James D. Suva - Citigroup Global Markets, Inc. (Broker)</p>
<p>Rod B. Hall - JPMorgan Securities LLC</p>

【问题讨论】:

  • 欢迎来到 SO。你能分享这个“html.parser”文件的示例数据吗?
  • 谢谢@Sampath!我用 html 代码更新了我的问题。

标签: python html python-3.x web-scraping beautifulsoup


【解决方案1】:

IIUC,一个非常粗略的解决方案可能是:

from bs4 import BeautifulSoup

s = '''
<div id="a-cont"><div class="p p1"></div><div class="sa-art article-width" id="a-body"><p>Apple, Inc. (NASDAQ:<a href="https://seekingalpha.com/symbol/AAPL" title="Apple Inc.">AAPL</a>)</p>
<p>Q4 2016 Earnings Call</p>
<p>October 25, 2016 5:00 pm ET</p>
<p><strong>Executives</strong></p>
<p>Nancy Paxton - Apple, Inc.</p>
<p>Timothy Donald Cook - Apple, Inc.</p>
<p>Luca Maestri - Apple, Inc.</p>
<p><strong>Analysts</strong></p>
<p>Eugene Charles Munster - Piper Jaffray &amp; Co.</p>
<p>Kathryn Lynn Huberty - Morgan Stanley &amp; Co. LLC</p>
<p>Shannon S. Cross - Cross Research LLC</p>
<p>Antonio M. Sacconaghi - Sanford C. Bernstein &amp; Co. LLC</p>
<p>Simona K. Jankowski - Goldman Sachs &amp; Co.</p>
<p>Steven M. Milunovich - UBS Securities LLC</p>
<p>Wamsi Mohan - Bank of America Merrill Lynch</p>
<p>James D. Suva - Citigroup Global Markets, Inc. (Broker)</p>
<p>Rod B. Hall - JPMorgan Securities LLC</p>
'''

bsobj = BeautifulSoup(s, "lxml")
res = []

for i in bsobj.find('strong').find_all_next('p'):
    if i.text == 'Analysts':
        break
    else:
        res.append(i.text)

res

你会得到:

['Nancy Paxton - Apple, Inc.',
 'Timothy Donald Cook - Apple, Inc.',
 'Luca Maestri - Apple, Inc.']

经过 OP 的进一步解释,最终的代码应该是这样的:

import textwrap
import os
from bs4 import BeautifulSoup

res = {}
directory ='C:/Research syntheses - Meta analysis/Transcripts/test/1/'
for filename in os.listdir(directory):
    if filename.endswith('.html'):
        fname = os.path.join(directory,filename)
        with open(fname, 'r') as f:
            soup = BeautifulSoup(f.read(),'html.parser')

            res[filename] = []
            for i in soup.find('strong').find_all_next('p'):
                if i.text == 'Analysts':
                    break
                else:
                    res[filename].append(i.text)

【讨论】:

  • 这也适用于我已经下载的文件吗? (如我的问题中所述)并使用代码“ import textwrap import os from bs4 import BeautifulSoup directory ='C:/test/out' for filename in os.listdir(directory): if filename.endswith('.html '): fname = os.path.join(directory,filename) with open(fname, 'r') as f: soup = BeautifulSoup(f.read(),'html.parser')"
  • 只要bsobj = BeautifulSoup(s, "lxml") 中的对象与我的代码中的对象具有相似的结构,我会说是的。就试一试吧。如果这个答案解决了你的问题,请采纳!
  • 我试图将我的“目录加载”与您的方法合并,但这似乎不起作用(在问题中,我编辑了我正在谈论的这个合并)。
  • 在您的代码中,删除 bsobj = BeautifulSoup(f.read(), "lxml"),将我的 for 循环移动到初始 for 循环内,将 bsobj 重命名为 soup
  • 我发现了问题,tekst 应该是“Analysts:” 我也可以按列或行打印吗?这个问题我会回答的,谢谢!!!
猜你喜欢
  • 1970-01-01
  • 2015-05-17
  • 1970-01-01
  • 2019-06-22
  • 2018-08-09
  • 1970-01-01
  • 2018-09-10
  • 1970-01-01
  • 1970-01-01
相关资源
最近更新 更多