编码 UTF-8 时出错

Question

我正在尝试从网站获取文本数据，但此代码显示了一些错误。请让我知道错误在哪里。

import requests

from bs4 import BeautifulSoup

def getportions(soup):

for p in soup.find_all("p", {"class": ""}):   
    yield p.text


def readpage(address):   
   page = requests.get(address)    
   soup = BeautifulSoup(page.text, "html.parser")
   output_text = ''
   for s in getportions(soup):
      output_text += s.encode("utf8")
      output_text += "\n"
   print (output_text)
   print ("End of article")
   fp = open("content.txt", "w")
   fp.write(output_text)
if __name__ == "__main__":
  readpage("http://yahoo.com")

错误如下图：

output_text += s.encode("utf8")。 TypeError: 无法将 'bytes' 对象隐式转换为 str

Answer 1

如果您使用 Python 3，所有字符串都是原生 unicode，您可以在打开文件时指定编码。你的代码可能变成：

def readpage(address):   
   ...
   output_text = ''
   for s in getportions(soup):
      output_text += s
      output_text += "\n"
   print (output_text)
   print ("End of article")
   fp = open("content.txt", "w", encoding='utf8')
   fp.write(output_text)

如果您只是想通过将所有非 ascii 字符替换为 ? 来清理文本，请以这种方式打开文件：

   fp = open("content.txt", "w", encoding='ascii', errors='replace')