如何在 python 中获取网页的前 3 个句子？答案

【问题标题】：How do I get the first 3 sentences of a webpage in python?如何在 python 中获取网页的前 3 个句子？
【发布时间】：2020-10-09 16:52:31
【问题描述】：

我有一个任务，我可以做的一件事是找到网页的前 3 个句子并显示它。查找网页文本很容易，但我无法弄清楚如何找到前 3 个句子。

import requests
from bs4 import BeautifulSoup

url = 'https://www.troyhunt.com/the-773-million-record-collection-1-data-reach/'
res = requests.get(url)
html_page = res.content
soup = BeautifulSoup(html_page, 'html.parser')
text = soup.find_all(text=True)

output = ''
blacklist = [
      '[document]',
      'noscript',
      'header',
      'html',
      'meta',
      'head',
      'input',
      'script'
]

for t in text:
  if (t.parent.name not in blacklist):
    output += '{} '.format(t)

tempout = output.split('.')
for i in range(tempout):
  if (i >= 3):
    tempout.remove(i)

output = '.'.join(tempout)

print(output)

【问题讨论】：

检查我的答案是否满足你的要求。

标签： python html list beautifulsoup

【解决方案1】：

很难从文本中找出句子。通常，您会寻找可能完成句子的字符，例如“。”和 '！'。但是句点（'.'）可以出现在句子的中间，例如人名的缩写。我使用正则表达式来查找后跟单个空格或字符串结尾的句点，这适用于前三个句子，但不适用于任意句子。

import requests
from bs4 import BeautifulSoup
import re

url = 'https://www.troyhunt.com/the-773-million-record-collection-1-data-reach/'
res = requests.get(url)
html_page = res.content
soup = BeautifulSoup(html_page, 'html.parser')

paragraphs = soup.select('section.article_text p')
sentences = []
for paragraph in paragraphs:
    matches = re.findall(r'(.+?[.!])(?: |$)', paragraph.text)
    needed = 3 - len(sentences)
    found = len(matches)
    n = min(found, needed)
    for i in range(n):
        sentences.append(matches[i])
    if len(sentences) == 3:
        break
print(sentences)

打印：

['Many people will land on this page after learning that their email address has appeared in a data breach I\'ve called "Collection #1".', "Most of them won't have a tech background or be familiar with the concept of credential stuffing so I'm going to write this post for the masses and link out to more detailed material for those who want to go deeper.", "Let's start with the raw numbers because that's the headline, then I'll drill down into where it's from and what it's composed of."]

【讨论】：

【解决方案2】：

要抓取前三个句子，只需将这些行添加到您的代码中：

section = soup.find('section',class_ = "article_text post") #Finds the section tag with class "article_text post"

txt = section.p.text #Gets the text within the first p tag within the variable section (the section tag)

print(txt)

输出：

Many people will land on this page after learning that their email address has appeared in a data breach I've called "Collection #1". Most of them won't have a tech background or be familiar with the concept of credential stuffing so I'm going to write this post for the masses and link out to more detailed material for those who want to go deeper.

希望这会有所帮助！

【讨论】：

【解决方案3】：

其实使用美化汤可以通过“article_text post”类过滤查看源码：

myData=soup.find('section',class_ = "article_text post")
print(myData.p.text)

并获取p元素的内部文本

用这个代替soup = BeautifulSoup(html_page, 'html.parser')

【讨论】：