【问题标题】:extract <label><span> tag on html with python用python在html上提取<label><span>标签
【发布时间】:2018-10-04 00:31:34
【问题描述】:

我想提取如下网页: https://www.glassdoor.com/Overview/Working-at-Apple-EI_IE1138.11,16.htm,所以我想将结果返回为以下格式。

Website       Headquarters  Size             Revenue                Type
www.apple.com Cupertino, CA 10000+ employees $10+ billion (USD) per year     Company - Public (AAPL)

然后我使用下面的代码和beatifulsoup 来得到这个。

all_href = com_soup.find_all('span', {'class': re.compile('value')})
all_href = list(set(all_href))

它返回带有&lt;span&gt; 的标签。此外,它没有在&lt;label&gt;下显示标签

[<span class="value"> Computer Hardware &amp; Software</span>,
 <span class="value"> Company - Public (AAPL) </span>,
 <span class="value">10000+ employees</span>,
 <span class="value"> $10+ billion (USD) per year</span>,
 <span class="value-title" title="4.0"></span>,
 <span class="value">Cupertino, CA</span>,
 <span class="value"> 1976</span>,
 <span class="value-title" title="5.0"></span>,
 <span class="value website"><a class="link" href="http://www.apple.com" rel="nofollow noreferrer" target="_blank">www.apple.com</a></span>]

【问题讨论】:

    标签: python beautifulsoup urllib


    【解决方案1】:

    你的beautifulsoup pull 太具体了。您正在捕获所有“跨度”标签,其中 class= 值。

    当您查看 HTML 时,您可以通过搜索某些字段的文本来快速找到该部分。您应该做的是获取 class= 'infoEntity' 的任何 div 标签内的所有内容,其中包含您有兴趣从“概述”部分获取的所有 7 个字段。

    在其中,每个字段都有一个标签标签,它具有与您想要的标签相关的属性,并且在概述部分中。

    所以,从以下开始:

    from bs4 import BeautifulSoup
    data = """
    <div class="eep-pill"><p class="tightVert h2 white"><strong>Enhanced</strong> Profile&nbsp;<span class="round ib"><i class="icon-star-white"></i></span></p></div></header><section class="center flex-grid padVertLg eepModal"><h2>Try Enhanced Profile Free for a Month</h2><p>Explore the many benefits of having a premium branded profile on Glassdoor, like increased influence and advanced analytics.</p><div class="margBot"><i class="feaIllustration"></i></div><a href='/employers/enhanced/landing_input.htm?src=info_mod' class='gd-btn gd-btn-link gradient gd-btn-1 gd-btn-med span-1-2'><span>Get Started</span><i class='hlpr'></i></a><p>Changes wont be saved until you sign up for an Enhanced Profile subscription.</p></section></div></article><article id='MainCol'><div id='EmpBasicInfo' class='module empBasicInfo ' data-emp-id='1138'><div class=''><header class='tbl fill '><h2 class='cell middle tightVert blockMob'> Apple Overview</h2></header><div class='info flexbox row col-hh'><div class='infoEntity'><label>Website</label><span class='value website'><a class="link" href="http://www.apple.com" target="_blank" rel="nofollow noreferrer">www.apple.com</a></span></div><div class='infoEntity'><label>Headquarters</label><span class='value'>Cupertino, CA</span></div><div class='infoEntity'><label>Size</label><span class='value'>10000+ employees</span></div><div class='infoEntity'><label>Founded</label><span class='value'> 1976</span></div><div class='infoEntity'><label>Type</label><span class='value'> Company - Public (AAPL) </span></div><div class='infoEntity'><label>Industry</label><span class='value'> Computer Hardware & Software</span></div><div class='infoEntity'><label>Revenue</label><span class='value'> $10+ billion (USD) per year</span></div></div></div><div class=''><div data-full="We&amp;rsquo;re a diverse collection of thinkers and doers, continually reimagining what&amp;rsquo;s possible to help us all do what we love in new ways. The people who work here have reinvented entire industries with the Mac, iPhone, iPad, and Apple Watch, as well as with services, including iTunes, the App Store, Apple Music, and Apple Pay. And the same passion for innovation that goes into our products also applies to our practices &amp;mdash; strengthening our commitment to leave the world better than we found it." class='margTop empDescription'> We&rsquo;re a diverse collection of thinkers and doers, continually reimagining what&rsquo;s possible to help us all do what we love in new ways. The people who work here have reinvented entire industries with the Mac, iPhone, iPad, and Apple Watch, as well as with ... <span class='link minor moreLink' id='ExpandDesc'>Read more</span></div><div class='hr'><hr/></div><h3 class='margTop'>Glassdoor Awards</h3>
    """
    items = []
    soup = BeautifulSoup(data, 'lxml')
    get_info = iter(soup.find_all("div", {"class" : "infoEntity"}))
    for item in get_info:
        label = item.find("label")
        value = item.find("span")
        items.append((label.string, value.string))
    

    这样,您将获得项目中的元组列表,打印为:

    [('Website', 'www.apple.com'), ('Headquarters', 'Cupertino, CA'), ('Size', '10000+ employees'), ('Founded', ' 1976'), ('Type', ' Company - Public (AAPL) '), ('Industry', ' Computer Hardware & Software'), ('Revenue', ' $10+ billion (USD) per year')]
    

    您可以从那里以您喜欢的任何格式打印该列表。

    【讨论】:

    • 我知道提取数据集的代码有什么问题。谢谢!
    【解决方案2】:

    正如我在https://www.glassdoor.com/Overview/Working-at-Apple-EI_IE1138.11,16.htm 中注意到的那样

    你应该找到&lt;div class="infoEntity"&gt;而不是&lt;span class="value"&gt;,这样才能得到你想要的。

    all_href = com_soup.find_all('div', {'class': re.compile('infoEntity')}).find_all(['span','label'])
    all_href = list(set(all_href))
    

    它将返回您想要的所有&lt;span&gt;&lt;label&gt;

    如果你想让&lt;span&gt;&lt;label&gt; 在一起怎么办,而不是将其更改为

    all_href = [x.decode_contents(formatter="html") for x in com_soup.find_all('div', {'class': re.compile('infoEntity')})]
    #or
    all_href = [[x.find('span'), x.find('label')] for x in com_soup.find_all('div', {'class': re.compile('infoEntity')})]
    

    【讨论】:

      猜你喜欢
      • 1970-01-01
      • 1970-01-01
      • 2023-03-10
      • 1970-01-01
      • 1970-01-01
      • 1970-01-01
      • 1970-01-01
      • 1970-01-01
      • 1970-01-01
      相关资源
      最近更新 更多