【发布时间】:2021-04-27 22:23:50
【问题描述】:
使用 bs4 find() 函数我没有得到任何结果,即使它存在于 html 中。我正在尝试使用类 tab_content 获取所有 div。我在此链接 https://sofad.qc.ca/index.php?id_product=464&controller=product&id_lang=1 上找到了这个。所以请建议我如何以正确的方式做到这一点。 这是代码:
from bs4 import BeautifulSoup as bs
import requests
url = 'https://sofad.qc.ca/index.php?id_category=78&controller=category&id_lang=1'
r = requests.get(url)
soup = bs(r.content, 'html.parser')
tb = soup.find_all('a', class_='product_img_link')
for item in tb:
link = item.get('href')
r = requests.get(link)
soup = bs(r.content, 'lxml')
try:
title = soup.find('h1', {'itemprop':'name'}).text
except:
title = ''
try:
price = soup.find('span', id='our_price_display').text
except:
price = ''
try:
img = soup.find('img', id='bigpic').get('src')
except:
img = ''
try:
dv = " ".join(soup.find('div', class_='rte').text.split())
except:
dv = ''
for dvv in soup.find_all('div', class_='tab_content'):
print(dvv)
【问题讨论】:
标签: python web-scraping data-mining