newsletter3k，在第一个“by”字之后的可见文本中查找作者姓名

Question

Newsletter3K 是一个很好的用于新闻内容提取的 python 库。它大部分效果很好 .我想在可见文本中的第一个“by”字之后提取名称。这是我的代码，效果不好，请高人帮忙：

import re
from newspaper import Config
from newspaper import Article
from newspaper import ArticleException
from bs4 import BeautifulSoup
from bs4.element import Comment
import urllib.request
USER_AGENT = 'Mozilla/5.0 (Macintosh;Intel Mac OS X 10.15; rv:78.0)Gecko/20100101   Firefox/78.0'
config = Config()
config.browser_user_agent = USER_AGENT
config.request_timeout = 10 
html1='https://saugeentimes.com/new-perspectives-a-senior-moment-food-glorious-food-part-2/'
article = Article(html1.strip(), config=config)
article.download()
article.parse()
soup = BeautifulSoup(article)
## I want to take only visible text
[s.extract() for s in soup(['style', 'script', '[document]', 'head', 'title'])]
visible_text = soup.getText()
for line in visible_text:
    # Capture one-or-more words after first (By or by) the initial match
    match = re.search(r'By (\S+)', line)

    # Did we find a match?
    if match:
        # Yes, process it to print 
        By = match.group(1)
        print('By {}'.format(By))`

Answer 1

这不是一个全面的答案，但您可以从中构建。添加其他源时，您将需要扩展此代码。就像我之前所说的，我的Newspaper3k overview document 有很多提取示例，所以请仔细阅读。

在使用 newspaper3k 尝试这些提取方法之后，正则表达式应该是最后的努力：

• article.authors
• 元标签
• json
• 汤

from newspaper import Config
from newspaper import Article
from newspaper.utils import BeautifulSoup

USER_AGENT = 'Mozilla/5.0 (Macintosh; Intel Mac OS X 10.15; rv:78.0) Gecko/20100101 Firefox/78.0'

config = Config()
config.browser_user_agent = USER_AGENT
config.request_timeout = 10

urls = ['https://saugeentimes.com/new-perspectives-a-senior-moment-food-glorious-food-part-2',
        'https://www.macleans.ca/education/what-college-students-in-canada-can-expect-during-covid',
        'https://www.cnn.com/2021/02/12/asia/india-glacier-raini-village-chipko-intl-hnk/index.html',
        'https://www.latimes.com/california/story/2021-02-13/wildfire-santa-cruz-boulder-creek-residents-fear-water'
        '-quality',
        'https://foxbaltimore.com/news/local/maryland-lawmakers-move-ahead-with-first-tax-on-internet-ads-02-13-2021']

for url in urls:
    try:
        article = Article(url, config=config)
        article.download()
        article.parse()
        author = article.authors
        if author:
            print(author)
        elif not author:
            soup = BeautifulSoup(article.html, 'html.parser')
            author_tag = soup.find(True, {'class': ['td-post-author-name', 'byline']}).find(['a', 'span'])
            if author_tag:
                print(author_tag.get_text().replace('By', '').strip())
            else:
                print('no author found')
    except AttributeError as e:
        pass