无法在 BeautifulSoup 中美化抓取的 html答案

【问题标题】：Cannot prettify scraped html in BeautifulSoup无法在 BeautifulSoup 中美化抓取的 html
【发布时间】：2010-01-07 16:53:48
【问题描述】：

我有一个小脚本，它使用urllib2 获取网站内容，找到所有链接标签，在顶部和底部附加一小段 HTML，然后我尝试美化它。它不断返回 TypeError: sequence item 1: expected string, Tag found。我环顾四周，我真的找不到问题。一如既往，任何帮助，非常感谢。

import urllib2
from BeautifulSoup import BeautifulSoup
import re

reddit = 'http://www.reddit.com'
pre = '<html><head><title>Page title</title></head>'
post = '</html>'
site = urllib2.urlopen(reddit)
html=site.read()
soup = BeautifulSoup(html)
tags = soup.findAll('a')
tags.insert(0,pre)
tags.append(post)
soup1 = BeautifulSoup(''.join(tags))
print soup1.prettify()

这是回溯：

Traceback (most recent call last): File "C:\Python26\bea.py", line 21, in <module>
        soup1 = BeautifulSoup(''.join(tags))
TypeError: sequence item 1: expected string, Tag found

【问题讨论】：

是的，这是回溯：回溯（最近一次调用最后一次）：文件“C:\Python26\bea.py”，第 21 行，在 soup1 = BeautifulSoup(''. join(tags)) TypeError: sequence item 1: expected string, Tag found

标签： python html parsing beautifulsoup

【解决方案1】：

这对我有用：

soup1 = BeautifulSoup(''.join(str(t) for t in tags))

这个 pyparsing 解决方案也提供了一些不错的输出：

from pyparsing import makeHTMLTags, originalTextFor, SkipTo, Combine

# makeHTMLTags defines HTML tag patterns for given tag string
aTag,aEnd = makeHTMLTags("A")

# makeHTMLTags by default returns a structure containing
# the tag's attributes - we just want the original input text
aTag = originalTextFor(aTag)
aEnd = originalTextFor(aEnd)

# define an expression for a full link, and use a parse action to
# combine the returned tokens into a single string
aLink = aTag + SkipTo(aEnd) + aEnd
aLink.setParseAction(lambda tokens : ''.join(tokens))

# extract links from the input html
links = aLink.searchString(html)

# build list of strings for output
out = []
out.append(pre)
out.extend(['  '+lnk[0] for lnk in links])
out.append(post)

print '\n'.join(out)

打印：

<html><head><title>Page title</title></head>
  <a href="http://www.reddit.com/r/pics/" >pics</a>
  <a href="http://www.reddit.com/r/reddit.com/" >reddit.com</a>
  <a href="http://www.reddit.com/r/politics/" >politics</a>
  <a href="http://www.reddit.com/r/funny/" >funny</a>
  <a href="http://www.reddit.com/r/AskReddit/" >AskReddit</a>
  <a href="http://www.reddit.com/r/WTF/" >WTF</a>
  .
  .
  .
  <a href="http://reddit.com/help/privacypolicy" >Privacy Policy</a>
  <a href="#" onclick="return hidecover(this)">close this window</a>
  <a href="http://www.reddit.com/feedback" >volunteer to translate</a>
  <a href="#" onclick="return hidecover(this)">close this window</a>
</html>

【讨论】：

【解决方案2】：

soup1 = BeautifulSoup(''.join(unicode(tag) for tag in tags))

【讨论】：

我添加了你的行，现在在 BeautifulSoup.py TypeError: expected string or buffer 中出现类型错误

【解决方案3】：

乔纳森的回答有点语法错误，这是正确的：

    soup1 = BeautifulSoup(''.join([unicode(tag) for tag in tags]))

【讨论】：

与其将其作为答案，不如在乔纳森的答案下发表评论更合适。