【问题标题】:Web scraping data with line breaks带有换行符的 Web 抓取数据
【发布时间】:2019-01-13 17:30:26
【问题描述】:

我正在尝试从这个网址中刮取数据:https://www.apple.com/ca/shop/browse/home/specialdeals/mac/macbook_pro/13

我正在尝试检索显示

的行

8GB 2133MHz LPDDR3 板载内存

16GB 2133MHz LPDDR3 板载内存

containers = soup.findAll('tr', {'class': 'product'}) 的每个容器中使用 BeautifulSoup。问题是它周围有换行符和多个换行符,这让我很难解析。我怎样才能找回这个?

【问题讨论】:

    标签: python web web-scraping beautifulsoup


    【解决方案1】:

    查看源代码,这里最好的选择是将BeautifulSoupregular expressions 结合起来:

    import requests
    from bs4 import BeautifulSoup
    import re
    
    url = "https://www.apple.com/ca/shop/browse/home/specialdeals/mac/macbook_pro/13"
    
    r = requests.get(url)
    soup = BeautifulSoup(r.text, 'lxml')
    for td in soup.select('td.specs'):
        m = re.search('^(8|16).*?onboard memory.*?$', td.text, flags=re.M|re.I)
        if not m:
            continue
        print(td.select_one('h3').text.strip())
        print('Full text: {} | Memory: {}'.format(m[0].strip(), m[1]))
        print('-' * 80)
    

    此代码查找所有 8 或 16 GB 的产品并打印它们:

    Refurbished 13.3-inch MacBook Pro 2.3GHz dual-core Intel Core i5 with Retina display - Space Grey
    Full text: 8GB of 2133MHz LPDDR3 onboard memory | Memory: 8
    --------------------------------------------------------------------------------
    Refurbished 13.3-inch MacBook Pro 2.3GHz dual-core Intel Core i5 with Retina display - Silver
    Full text: 8GB of 2133MHz LPDDR3 onboard memory | Memory: 8
    --------------------------------------------------------------------------------
    Refurbished 13.3-inch MacBook Pro 2.0GHz Dual-core Intel Core i5 with Retina Display — Space Grey
    Full text: 8GB of 1866MHz LPDDR3 onboard memory | Memory: 8
    --------------------------------------------------------------------------------
    Refurbished 13.3-inch MacBook Pro 2.3GHz dual-core Intel Core i5 with Retina display - Silver
    Full text: 8GB of 2133MHz LPDDR3 onboard memory | Memory: 8
    --------------------------------------------------------------------------------
    Refurbished 13.3-inch MacBook Pro 2.3GHz dual-core Intel Core i5 with Retina display - Space Grey
    Full text: 8GB of 2133MHz LPDDR3 onboard memory | Memory: 8
    --------------------------------------------------------------------------------
    Refurbished 13.3-inch Macbook Pro 2.9GHz Dual-core Intel Core i5 with Retina Display - Space Grey
    Full text: 8GB of 2133MHz LPDDR3 onboard memory | Memory: 8
    --------------------------------------------------------------------------------
    Refurbished 13.3-inch Macbook Pro 2.9GHz Dual-core Intel Core i5 with Retina Display - Silver
    Full text: 8GB of 2133MHz LPDDR3 onboard memory | Memory: 8
    --------------------------------------------------------------------------------
    Refurbished 13.3-inch Macbook Pro 2.9GHz Dual-core Intel Core i5 with Retina Display - Silver
    Full text: 8GB of 2133MHz LPDDR3 onboard memory | Memory: 8
    --------------------------------------------------------------------------------
    Refurbished 13.3-inch MacBook Pro 3.1GHz dual-core Intel Core i5 with Retina display - Silver
    Full text: 8GB of 2133MHz LPDDR3 onboard memory | Memory: 8
    --------------------------------------------------------------------------------
    Refurbished 13.3-inch MacBook Pro 3.1GHz dual-core Intel Core i5 with Retina display - Space Grey
    Full text: 8GB of 2133MHz LPDDR3 onboard memory | Memory: 8
    --------------------------------------------------------------------------------
    Refurbished 13.3-inch Macbook Pro 3.3GHz Dual-core Intel Core i7 with Retina Display - Space Grey
    Full text: 16GB of 2133MHz LPDDR3 onboard memory | Memory: 16
    --------------------------------------------------------------------------------
    

    【讨论】:

    猜你喜欢
    • 1970-01-01
    • 1970-01-01
    • 1970-01-01
    • 1970-01-01
    • 2022-07-22
    • 1970-01-01
    • 1970-01-01
    • 1970-01-01
    • 1970-01-01
    相关资源
    最近更新 更多