美丽的汤链接 - href 不返回任何结果

Question

我正在尝试仅检索以下公司页面上的链接： https://clutch.co/it-services/msp

看来这是一个常见问题，我花了一天时间查看其他帖子，但没有任何成功。

代码：

links = []
for l in soup.find_all(class_='website-link website-link-a'):
    results = (l.get('href'))
    links.append(results)

print(links)

输出：

[None, None, None, None, None, None, None, None, None, None, None, None, None, None, None, None, None, None, None, None]

当我只打印 soup.find_all 的结果时，我得到：

<a data-extlink-pid="1219089" href="https://fulcrumdigital.com/" rel="nofollow" target="_blank">
<i class="icon icon-visit-site"></i><span class="">Visit Website</span>
</a>
</li>, etc, etc,

我需要在 href 之后提取但不知道如何。非常感谢任何建议。

Answer 1

您可以使用 CSS 选择器'.website-link-a > a'（在带有class="website-link-a" 的标签下直接选择每个<a> 标签）：

import requests
from bs4 import BeautifulSoup

url = 'https://clutch.co/it-services/msp'
soup = BeautifulSoup(requests.get(url).content, 'html.parser')

for a in soup.select('.website-link-a > a'):
    print(a['href'])

打印：

http://electric.ai/
http://www.symphony-solutions.com/?utm_source=clutch.co&utm_medium=referral&utm_campaign=it-services-msp
https://www.bairesdev.com/?utm_source=clutch.co&utm_medium=referral&utm_campaign=msp
https://www.helixstorm.com/?utm_source=clutch.co&utm_medium=referral&utm_campaign=it-services-msp
http://www.sundevs.com/?utm_source=clutch.co&utm_medium=referral&utm_campaign=it-services-msp
http://www.computersolutionseast.com/?utm_source=clutch.co&utm_medium=referral&utm_campaign=it-services-msp
/your-project
http://techmd.com
http://www.sugarshot.io/?utm_source=clutch.co&utm_medium=referral&utm_campaign=directory
https://www.empist.com?utm_source=clutch.co&utm_medium=referral
http://www.frameworkIT.com/?utm_source=clutch.co&utm_medium=referral
https://www.clickittech.com/
https://cyberduo.com/?utm_source=clutch.co&utm_medium=referral&utm_campaign=it-services-msp
http://www.realnets.com/?utm_source=clutch.co&utm_medium=referral
https://www.ibexlabs.com/?utm_source=clutch.co&utm_medium=referral
https://bianor.com/
http://www.endpoint.com/?utm_source=clutch.co&utm_medium=referral
https://devopsprodigy.com/?utm_source=clutch.co&utm_medium=referral&utm_campaign=directory
https://vrpconsulting.com/
https://siliconreef.co.uk/?utm_source=clutch.co&utm_medium=referral
http://www.agencypartner.com?utm_source=clutch&utm_medium=profile&utm_campaign=directory_listing

Answer 2

您需要对find_all 方法进行一些更改。您应该使用soup.find_all('li',class_='website-link website-link-a') 而不是soup.find_all(class_='website-link website-link-a')。您需要更改 l.a.get('href') 而不是 l.get('href')。然后你得到你的实际结果。您可以尝试获取链接：

import requests
from bs4 import BeautifulSoup
url = 'https://clutch.co/it-services/msp'
response = requests.get(url)
soup = BeautifulSoup(response.content, 'html.parser')

links = []
for l in soup.find_all('li',class_='website-link website-link-a'):
    results = (l.a.get('href'))
    links.append(results)

print(links)

输出将是：

['http://electric.ai/', 'http://www.symphony-solutions.com/?utm_source=clutch.co&utm_medium=referral&utm_campaign=it-services-msp', 'https://www.bairesdev.com/?utm_source=clutch.co&utm_medium=referral&utm_campaign=msp', 'https://www.helixstorm.com/?utm_source=clutch.co&utm_medium=referral&utm_campaign=it-services-msp', 'http://www.sundevs.com/?utm_source=clutch.co&utm_medium=referral&utm_campaign=it-services-msp', 'http://www.computersolutionseast.com/?utm_source=clutch.co&utm_medium=referral&utm_campaign=it-services-msp', 'http://techmd.com', 'http://www.sugarshot.io/?utm_source=clutch.co&utm_medium=referral&utm_campaign=directory', 'https://www.empist.com?utm_source=clutch.co&utm_medium=referral', 'http://www.frameworkIT.com/?utm_source=clutch.co&utm_medium=referral', 'https://www.clickittech.com/', 'https://cyberduo.com/?utm_source=clutch.co&utm_medium=referral&utm_campaign=it-services-msp', 'http://www.realnets.com/?utm_source=clutch.co&utm_medium=referral', 'https://www.ibexlabs.com/?utm_source=clutch.co&utm_medium=referral', 'https://bianor.com/', 'http://www.endpoint.com/?utm_source=clutch.co&utm_medium=referral', 'https://devopsprodigy.com/?utm_source=clutch.co&utm_medium=referral&utm_campaign=directory', 'https://vrpconsulting.com/', 'https://siliconreef.co.uk/?utm_source=clutch.co&utm_medium=referral', 'http://www.agencypartner.com?utm_source=clutch&utm_medium=profile&utm_campaign=directory_listing']