【问题标题】:customize BeautifulSoup's prettify by tag自定义 BeautifulSoup 的 prettify by tag
【发布时间】:2013-07-09 03:17:04
【问题描述】:

我想知道是否有可能使prettify 不在特定标签上创建新行。

我想让spana标签不会分开,例如:

doc="""<div><div><span>a</span><span>b</span>
<a>link</a></div><a>link1</a><a>link2</a></div>"""

from bs4 import BeautifulSoup as BS
soup = BS(doc)
print soup.prettify()

下面是我要打印的内容:

<div>
    <div>
        <span>a</span><span>b</span>
        <a>link</a>
    </div>
    <a>link1</a><a>link2</a>
</div>

但这就是实际打印的内容:

<div>
    <div>
        <span>
            a
        </span>
        <span>
            b
        </span>
        <a>
            link
        </a>
    </div>
    <a>
        link1
    </a>
    <a>
        link2
    </a>
</div>

在这样的新行上放置内联样式的标签实际上会在它们之间增加空间,稍微改变实际页面的外观。我会将您链接到显示差异的两个 jsfiddles:

anchor tags on new lines

anchor tags next to eachother

如果你想知道为什么这对 BeautifulSoup 很重要,那是因为我正在编写一个网页调试器,而 prettify 函数会非常有用(以及 bs4 中的其他东西)。但是,如果我美化文档,那么我就有可能改变一些东西。

那么,有什么方法可以自定义prettify 函数,以便我可以将其设置为不分解某些标签?

【问题讨论】:

    标签: python html beautifulsoup


    【解决方案1】:

    在找不到更好的解决方案时,我发布了一个快速破解方法。

    我实际上是在我的项目中使用它来避免破坏 textareas 和 pre 标签。将 ['span', 'a'] 替换为要防止缩进的标签。

    markup = """<div><div><span>a</span><span>b</span>
    <a>link</a></div><a>link1</a><a>link2</a></div>"""
    
    # Double curly brackets to avoid problems with .format()
    stripped_markup = markup.replace('{','{{').replace('}','}}')
    
    stripped_markup = BeautifulSoup(stripped_markup)
    
    unformatted_tag_list = []
    
    for i, tag in enumerate(stripped_markup.find_all(['span', 'a'])):
        unformatted_tag_list.append(str(tag))
        tag.replace_with('{' + 'unformatted_tag_list[{0}]'.format(i) + '}')
    
    pretty_markup = stripped_markup.prettify().format(unformatted_tag_list=unformatted_tag_list)
    
    print pretty_markup
    

    【讨论】:

    • 正是我想要的!!!此外,“11”是这一天......它只有一个月大:D
    • 感谢您提供的信息! :D 我将编辑帖子以删除我对发布日期的评论。
    • 如果原始标记包含 javascript(实际上是括号),这将不起作用。这一点不足为奇,在使用format 时会产生KeyErrors。
    • 很好看,史蒂夫。我添加了一行来处理这个问题。
    • @IvanChaer 在开始替换它们后,我需要在末尾添加另一个pretty_markup = pretty_markup.replace('{{','{').replace('}}','}') 以使括号恢复正常。
    【解决方案2】:

    简短的回答是否定的。

    更长的答案并不容易。

    我仍在使用 bs3,所以这是针对 bs3 的 hack。我正在将它移植到 bs4。

    它本质上涉及子类化 Tag 和 BeautifulSoup 并重载 prettify(和相关)方法。

    代码:

    import sys
    import BeautifulSoup
    
    class Tag(BeautifulSoup.Tag):
        def __str__(self, encoding=BeautifulSoup.DEFAULT_OUTPUT_ENCODING,
                prettyPrint=False, indentLevel=0, pprint_exs=[]):
            """Returns a string or Unicode representation of this tag and
            its contents. To get Unicode, pass None for encoding.
    
            NOTE: since Python's HTML parser consumes whitespace, this
            method is not certain to reproduce the whitespace present in
            the original string."""
    
            encodedName = self.toEncoding(self.name, encoding)
    
            unflatten_here = (not self.name in pprint_exs)
    
            attrs = []
            if self.attrs:
                for key, val in self.attrs:
                    fmt = '%s="%s"'
                    if isinstance(val, basestring):
                        if self.containsSubstitutions and '%SOUP-ENCODING%' in val:
                            val = self.substituteEncoding(val, encoding)
    
                        # The attribute value either:
                        #
                        # * Contains no embedded double quotes or single quotes.
                        #   No problem: we enclose it in double quotes.
                        # * Contains embedded single quotes. No problem:
                        #   double quotes work here too.
                        # * Contains embedded double quotes. No problem:
                        #   we enclose it in single quotes.
                        # * Embeds both single _and_ double quotes. This
                        #   can't happen naturally, but it can happen if
                        #   you modify an attribute value after parsing
                        #   the document. Now we have a bit of a
                        #   problem. We solve it by enclosing the
                        #   attribute in single quotes, and escaping any
                        #   embedded single quotes to XML entities.
                        if '"' in val:
                            fmt = "%s='%s'"
                            if "'" in val:
                                # TODO: replace with apos when
                                # appropriate.
                                val = val.replace("'", "&squot;")
    
                        # Now we're okay w/r/t quotes. But the attribute
                        # value might also contain angle brackets, or
                        # ampersands that aren't part of entities. We need
                        # to escape those to XML entities too.
                        val = self.BARE_AMPERSAND_OR_BRACKET.sub(self._sub_entity, val)
    
                    attrs.append(fmt % (self.toEncoding(key, encoding),
                                        self.toEncoding(val, encoding)))
            close = ''
            closeTag = ''
            if self.isSelfClosing:
                close = ' /'
            else:
                closeTag = '</%s>' % encodedName
    
            prev = self.findPrevious(lambda x: isinstance(x, Tag))
            prev_sib = self.findPreviousSibling(lambda x: isinstance(x, Tag))
            ex_break_detected = (self.name != prev_sib.name) if(prev_sib and prev_sib.name in pprint_exs) else False
            break_detected = (self.name != prev.name) if(prev) else False
    
            indentTag, indentContents = 0, 0
            if prettyPrint:
                if(break_detected or unflatten_here):
                    indentContents = indentLevel + 1
                indentTag = indentLevel
                space = (' ' * (indentTag-1))
            contents = self.renderContents(encoding, prettyPrint, indentContents, pprint_exs, unflatten_here)
            if self.hidden:
                s = contents
            else:
                s = []
                attributeString = ''
                if attrs:
                    attributeString = ' ' + ' '.join(attrs)
                if prettyPrint and ex_break_detected and not unflatten_here:
                    s.append("\n")
                if prettyPrint and (unflatten_here or break_detected):
                    s.append(space)
                s.append('<%s%s%s>' % (encodedName, attributeString, close))
                if prettyPrint and unflatten_here:
                    s.append("\n")
                s.append(contents)
                if prettyPrint and contents and contents[-1] != "\n" and unflatten_here:
                    s.append("\n")
                if prettyPrint and closeTag and unflatten_here:
                    s.append(space)
                s.append(closeTag)
                if prettyPrint and closeTag and self.nextSibling and unflatten_here:
                    s.append("\n")
                if prettyPrint and isinstance(self.nextSibling, Tag) and self.nextSibling.name != self.name and not unflatten_here:
                    s.append("\n")
    
                s = ''.join(s)
            return s
    
        def renderContents(self, encoding=BeautifulSoup.DEFAULT_OUTPUT_ENCODING,
                           prettyPrint=False, indentLevel=0, pprint_exs=[], unflatten=True):
            """Renders the contents of this tag as a string in the given
            encoding. If encoding is None, returns a Unicode string.."""
            s=[]
            for c in self:
                text = None
                if isinstance(c, BeautifulSoup.NavigableString):
                    text = c.__str__(encoding)
                elif isinstance(c, Tag):
                    s.append(c.__str__(encoding, prettyPrint, indentLevel, pprint_exs))
                if text and prettyPrint:
                    text = text.strip()
                if text:
                    if prettyPrint and unflatten:
                        s.append(" " * (indentLevel-1))
                    s.append(text)
                    if prettyPrint and unflatten:
                        s.append("\n")
            return ''.join(s)
    BeautifulSoup.Tag = Tag
    
    class BeautifulStoneSoup(Tag, BeautifulSoup.BeautifulStoneSoup):
        pass
    BeautifulSoup.BeautifulStoneSoup = BeautifulStoneSoup
    
    class PumpkinSoup(BeautifulStoneSoup, BeautifulSoup.BeautifulSoup):
        def __init__(self, *args, **kwargs):
            self.pprint_exs = kwargs.pop("pprint_exs", [])
            super(BeautifulSoup.BeautifulSoup, self).__init__(*args, **kwargs)
        def prettify(self, encoding=BeautifulSoup.DEFAULT_OUTPUT_ENCODING):
            return self.__str__(encoding, True, pprint_exs=self.pprint_exs)
    
    doc = \
    '''
    <div>
     <div>
    <span>a</span><span>b</span>
      <a>link1</a>
      <a>link2</a>
    <span>c</span>
     </div>
    <a>link3</a><a>link4</a>
    </div>
    '''
    
    soup = PumpkinSoup(doc, pprint_exs = ["a", "span"])
    print soup.prettify()
    

    【讨论】:

      猜你喜欢
      • 2019-03-20
      • 2013-03-08
      • 1970-01-01
      • 1970-01-01
      • 1970-01-01
      • 1970-01-01
      • 2012-07-12
      • 1970-01-01
      • 1970-01-01
      相关资源
      最近更新 更多