如何通过 BeautifulSoup 获取已发布的博客文本？

Question

我制作了一个 Telegram 机器人并使用插件来获取博客（博主）的文本。但我收到以下错误。

代码：

import requests
import re
from bs4 import BeautifulSoup
from pyrogram import Client, filters, enums

@Client.on_message(filters.command("mzl"))
async def mzlyc(client, message):
    await client.send_chat_action(message.chat.id, action=enums.ChatAction.TYPING)
    mzr = message.text.split(None, 1)[1]
    mizoly = mzr.replace(" ", "+") if len(message.command) > 2 else mzr
    mizl = f"https://www.mizolyric.com/search?q={mizoly}&m=1"
    zol = requests.get(mizl).text
    soup = BeautifulSoup(zol, 'html.parser')
    item = soup.select_one("div[id^='post-body-']").text
    if not item:
        return await client.send_message(message.chat.id, \
            text="I lyrics duh hi ka zawng hmu zo lo.",  \
            reply_to_message_id=message.id)
    itms = item.replace("Continue Reading", " ") if "Continue Reading" in item else item
    await client.send_message(message.chat.id, text=f"{itms}", reply_to_message_id=message.id)

此代码适用于此模板：https://www.templatemark.com/2017/10/news16-magazine-blogger-template.html;，但不适用于模板 https://www.templatemark.com/2020/05/msd-responsive-grid-style-blogger-template.html。

错误：

Traceback (most recent call last):
2022-08-21T19:21:40.630002+00:00 app[worker.1]:   File "/app/.heroku/python/lib/python3.10/site-packages/pyrogram/dispatcher.py", line 240, in handler_worker
2022-08-21T19:21:40.630002+00:00 app[worker.1]:     await handler.callback(self.client, *args)
2022-08-21T19:21:40.630002+00:00 app[worker.1]:   File "/app/plugins/Tools4/mzlyrics.py", line 17, in mzlyc
2022-08-21T19:21:40.630003+00:00 app[worker.1]:     item = soup.select_one("div[id^='post-body-']").text
2022-08-21T19:21:40.630003+00:00 app[worker.1]: AttributeError: 'NoneType' object has no attribute 'text'
    

如何解决这个问题？

Answer 1

尝试以下更改：

zol = requests.get(mizl).content
soup = BeautifulSoup(zol.decode('utf-8','ignore'), 'html.parser')

这是encoding unicode characters 的问题here解释。