在没有用 python 美丽的汤挑选 br 标记后，网络抓取一些信息的困难

Question

所以我一直在尝试通过网络抓取公共网络目录https://www.kenyaplex.com/business-directory/?start=1&categoryid=286 以获取公司的联系方式。我的代码运行良好，但它没有在 br 标记之后选择信息，例如它没有在美丽汤的以下详细信息中选择电子邮件

<div class="c-detail">
<a href="https://www.kenyaplex.com/business-directory/86129-water-well-drilling-rig-gujarat-india.aspx">water well drilling rig, Gujarat, India</a><br/>+91-9825005407<br/>info@dhirajrigs.com</div>,

我试图更改列表的索引，但它不起作用，我附上了我所拥有的代码给我的图像。 我希望它同时选择电话号码和电子邮件，但它要么选择电话号码要么选择列出两者的电子邮件 我正在使用的代码如下

import requests
from bs4 import BeautifulSoup
import time

#DEFINING THE FIRST WEBPAGE
num = 1
#STRING FORMATTING THE URL TO CAPTURE DIFFRENT PAGES
url = 'https://www.kenyaplex.com/business-directory/?start={}&categoryid=286'.format(num)
#DEIFING THE BROWSER HEADERS SO THAT CAN WORK ON IT WITHOUT ERRORS
headers = {'User-Agent':'Chrome'}
records = []
while num < 100:
    url = 'https://www.kenyaplex.com/business-directory/?start={}&categoryid=286'.format(num)
    time.sleep(1)
    num += 30
    response = requests.get(url,headers)
    soup = BeautifulSoup(response.text,'html.parser')
    company_info = soup.find_all('div', attrs={'class':'c-detail'})
    print(company_info)
    #EXTRACTING SPECIFIC RECORDS    
    for name in school_info:
        Name_of_The_Company = name.find('a').text
        #Location_of_The_School = name.contents[2][2:]
        Contact_of_The_Company = name.contents[3]
        Email = name.contents[4]
        
        
        
        #converting the records to a tuple
        records.append((Name_of_The_Company,
                        #Location_of_The_School,
                        Contact_of_The_Company,
                       Email))
#EXPORTING TO A PANDAS FILE    
import pandas as pd
df = pd.DataFrame(records, columns = ['Name of The Company',
                                      #'Location of The School',
                                      'Contact of The Company',
                                     'Email'])
df.to_csv('boreholes.csv', index = False, encoding = 'utf-8')

Answer 1

要获取电话号码和电子邮件，您可以从详细信息中提取<a> 标签，然后获取整个文本 -> 拆分它 -> 使用列表推导进行基本过滤。

例如：

import requests
import pandas as pd
from bs4 import BeautifulSoup


url = 'https://www.kenyaplex.com/business-directory/?start=1&categoryid=286'
soup = BeautifulSoup(requests.get(url).content, 'html.parser')

records = []
for detail in soup.select('.c-detail'):
    a = detail.select_one('a')
    name, url = a.text, a['href']
    a.extract()
    contacts = detail.get_text(strip=True, separator='|').split('|')
    phones = [c for c in contacts if '@' not in c]
    emails = [c for c in contacts if '@' in c]
    records.append([name, url, phones[0] if phones else '',  emails[0] if emails else ''])


df = pd.DataFrame(records, columns = ['Name of The Company',
                                      'URL',
                                      'Contact of The Company',
                                     'Email'])

df.to_csv('data.csv', index = False, encoding = 'utf-8')

保存data.csv（来自 LibreOffice 的屏幕截图）：