使用 Beautiful soup 查找公司名称和地址的电话号码

Question

我有一个脚本，它可以在网站上抓取西班牙公司的名称、地区和省份。 html 中有另一个链接，它会将您带到包含电话号码的页面，但是当我尝试抓取 html 时，它会打印“none”。有没有办法让脚本自动移动到页面，把数字刮下来和公司行匹配？

import requests
from googlesearch import search
from bs4 import BeautifulSoup
for page in range(1,65):
    url = "https://www.expansion.com/empresas-de/ganaderia/granjas-en-general/{page}.html".format(page =page)

    page = requests.get(url)
    soup = BeautifulSoup(page.content, "html.parser")
    lists = soup.select("div#simulacion_tabla ul")

#scrape the list
    for lis in lists:
        title = lis.find('li', class_="col1").text
        location = lis.find('li', class_="col2").text
        province = lis.find('li', class_="col3").text
        link = lis.find('href', class_ ="col1")
        info = [title, location, province, link]
print(info)

或者，有没有办法使用 googlesearch 库来做到这一点？

非常感谢

Answer 1

第一个网址"https://www.expansion.com/empresas-de/ganaderia/granjas-en-general/index.html"不是

"https://www.expansion.com/empresas-de/ganaderia/granjas-en-general/1.html"

因此，您的脚本不会返回输出。

你可以这样试试

import requests
# from googlesearch import search
from bs4 import BeautifulSoup
baseurl = ["https://www.expansion.com/empresas-de/ganaderia/granjas-en-general/index.html"]

urls = [f'https://www.expansion.com/empresas-de/ganaderia/granjas-en-general/{i}.html'.format(i) for i in range(2,5)]


allurls = baseurl + urls
print(allurls)


for url in allurls:

    page = requests.get(url)
    soup = BeautifulSoup(page.content, "html.parser")
    lists = soup.select("div#simulacion_tabla ul")

    #scrape the list
    for lis in lists:
        title = lis.find('li', class_="col1").text
        location = lis.find('li', class_="col2").text
        province = lis.find('li', class_="col3").text
        link = lis.select("li.col1 a")[0]['href']
        info = [title, location, province, link]
        print(info)