【问题标题】:How to get href from an a tag inside a div如何从div中的a标签获取href
【发布时间】:2015-12-04 18:54:53
【问题描述】:

我正在使用beautifulsoup,有些我无法在a 标签中提取href,无论我做什么,它都会向我返回错误。这是我正在使用的功能

def scrape_a(url):
  r = requests.get(url)
  soup = BeautifulSoup(r.content)
  news =  soup.find_all("div", attrs={"class": "news"})
  return news

html数据结构是

<div class="news">
<a href="www.link.com">
<h2 class="heading">
Kenyan police foil potential bomb attack in Nairobi mall 
</h2>
<div class="teaserImg">
<img alt="" border="0" height="124" src="/image">
</div>
<p> text </p>
</a>
</div>

我想从中提取的是 href 和 h2 class='heading',每当我尝试获取这两者时,我都会得到一个错误,none type object has no attribute get item

【问题讨论】:

    标签: python tags beautifulsoup


    【解决方案1】:

    这样的事情怎么样?

    from bs4 import BeautifulSoup
    
    def get_news_class_hrefs(html):
        """
        Finds all urls pointed to by all links inside
        'news' class div elements
        """
        soup = BeautifulSoup(html, 'html.parser')
        links = [a['href'] for div in soup.find_all("div", attrs={"class": "news"}) for a in div.find_all('a')]
        return links
    
    # example html copied from question
    html="""<div class="news">
    <a href="www.link.com">
    <h2 class="heading">
    Kenyan police foil potential bomb attack in Nairobi mall 
    </h2>
    <div class="teaserImg">
    <img alt="" border="0" height="124" src="/image">
    </div>
    <p> text </p>
    </a>"""
    
    get_news_class_hrefs(html)
    # Output:
    # [u'www.link.com']
    

    【讨论】:

    • 返回错误:返回 self.attrs[key] KeyError: 'href'
    • 嗯...无法重现该问题?你到底在跑什么?我将更新以显示我正在运行的完整代码。
    • 哦,如果您使用的是 BeautifulSoupBeautifulSoup(html) 并使用 findAll 而不是 find_all -- soup.find_all() 在旧版本中将始终返回 None版本
    猜你喜欢
    • 2010-10-27
    • 2021-12-26
    • 1970-01-01
    • 1970-01-01
    • 2017-06-04
    • 1970-01-01
    • 1970-01-01
    • 1970-01-01
    • 2016-01-09
    相关资源
    最近更新 更多