如何仅解析 Python 网页中的链接？答案

【问题标题】：How do I parse only the links from a webpage in Python?如何仅解析 Python 网页中的链接？
【发布时间】：2014-11-30 04:54:31
【问题描述】：

links = re.findall(r'\w+://\w+.\w+.\w+\w+\w.+"', page)

解析网页中的链接。

如有任何帮助，我们将不胜感激。这是我解析http://www.soc.napier.ac.uk/~cs342/CSN08115/cw_webpage/index.html得到的：

        #my current output#
        http://net.tutsplus.com/tutorials/other/8-regular-expressions-you-should-know/"
        http://www.asecuritysite.com/content/icon_clown.gif" alt="if broken see alex@school.ac.uk +44(0)1314552759" height="100"
        http://www.rottentomatoes.com/m/sleeper/"
        http://www.rottentomatoes.com/m/sleeper/trailer/"
        http://www.rottentomatoes.com/m/star_wars/"
        http://www.rottentomatoes.com/m/star_wars/trailer/"
        http://www.rottentomatoes.com/m/wargames/"
        http://www.rottentomatoes.com/m/wargames/trailer/"
        https://www.sans.org/press/sans-institute-and-crowdstrike-partner-to-offer-hacking-exposed-live-webinar-series.php"> SANS to Offer "Hacking Exposed Live"
        https://www.sans.org/webcasts/archive/2013"

        #I want to get this when i run the module#
        http://net.tutsplus.com/tutorials/other/8-regular-expressions-you-should-know/
        http://www.asecuritysite.com/content/icon_clown.gif
        http://www.rottentomatoes.com/m/sleeper/
        http://www.rottentomatoes.com/m/sleeper/trailer/
        http://www.rottentomatoes.com/m/star_wars/
        http://www.rottentomatoes.com/m/star_wars/trailer/
        http://www.rottentomatoes.com/m/wargames/
        http://www.rottentomatoes.com/m/wargames/trailer/
        https://www.sans.org/press/sans-institute-and-crowdstrike-partner-to-offer-hacking-exposed-live-webinar-series.php
        https://www.sans.org/webcasts/archive/2013

【问题讨论】：

重复stackoverflow.com/questions/1080411/…

标签： python html regex html-parsing

【解决方案1】：

You should not use regular expressions for parsing HTML. 有专门的工具叫做HTML 解析器。

这是一个使用BeautifulSoup 和requests 的示例：

from bs4 import BeautifulSoup
import requests

page = requests.get('http://www.soc.napier.ac.uk/~cs342/CSN08115/cw_webpage/index.html')
soup = BeautifulSoup(page.content)

for link in soup.find_all('a', href=True):
    print link.get('href')

打印：

http://www.rottentomatoes.com/m/sleeper/
http://www.rottentomatoes.com/m/sleeper/trailer/
http://www.rottentomatoes.com/m/wargames/
http://www.rottentomatoes.com/m/wargames/trailer/
...

【讨论】：

【解决方案2】：

\w+://\w+\.\w+\.\w+[^"]+

试试这个。查看演示。

http://regex101.com/r/hQ9xT1/31

【讨论】：

我很感激..真的很感激

【解决方案3】：

通过 Beautifulsoup CSS selectors.

>>> from bs4 import BeautifulSoup
>>> import requests
>>> page = requests.get('http://www.soc.napier.ac.uk/~cs342/CSN08115/cw_webpage/index.html')
>>> soup = BeautifulSoup(page.content)
>>> for i in soup.select('a[href]'):
        print(i['href'])

http://www.rottentomatoes.com/m/sleeper/
http://www.rottentomatoes.com/m/sleeper/trailer/
http://www.rottentomatoes.com/m/wargames/
http://www.rottentomatoes.com/m/wargames/trailer/
..................

【讨论】：