Python 美汤 find_all

Question

您好，我正在尝试从网站获取一些信息。如果我的格式有任何错误，请原谅我这是我第一次在 SO 上发帖。

soup.find('div', {"class":"stars"}) 

从这里我收到了

<div class="stars" title="4.0 star rating">
<i class="star star--large star-0"></i><i class="star star--large star- 
1"></i><i class="star star--large star-2"></i><i class="star star--large 
star-3"></i><i class="star star--large star-4 star--large--muted"></i> 
</div>

我需要那个"4.0 star rating"

当我使用时：

soup.find('div', {"class":"stars"})["title"]

它有效，但不适用于 find_all。但我试图找到所有案例并将它们放入列表中。

这是我下面的完整代码。

    def get_info():
        from IPython.display import HTML
        import requests
        from bs4 import BeautifulSoup
        n = 1
        for page in range(53):
            url = f"https://www.sitejabber.com/reviews/apple.com?page= 
   {n}&sort=Reviews.processed&direction=DESC#reviews"
            r = requests.get(url)
            soup = BeautifulSoup(r.text, 'lxml')
            all_reviews = soup.find_all('div', {'class':"truncate_review"})
            all_dates = soup.find_all('div', {'class':'review__date'},'title')
            all_titles = soup.find_all('span', {'class':'review__title__text'})
            reviews_class = soup.find('div', {"class":"review__stars"})
            for review in all_reviews:

    all_reviews_list.append(review.text.replace("\n","").replace("\t",""))
            for date in all_dates:

all_dates_list.append(date.text.replace("\n","").replace("\t",""))
            for title in all_titles:

  all_titles_list.append(title.text.replace("\n","").replace("\t",""))
            for stars in reviews_class.find_all('div', {'class':'stars'}):
                all_star_ratings.append(stars['title'])



            n += 1

对不起，我的缩进有点乱，但这是我的完整代码。

Answer 1

像在字典中一样遍历 bs4 元素。
如果您使用的是find()：

soup.find('div', {"class":"stars"}) ['title']

这有效，因为find() 返回单个值。
但是如果你使用find_all()，它会返回一个列表并且list[string]是一个无效的进程。
因此，您可以创建一个列表：

res = []
for i in soup.find_all('div', {"class":"stars"}):
    res.append(i['title'])

否则，作为单行：

res = [i['title'] for i in soup.find_all('div', {"class":"stars"})]

---

由于要review的所有title，所以需要指定review容器，即scrape from:

<div class="review__container">

所以代码是：

review = soup.find_all('div',class_="review__container")
res = [i['title'] for j in review for i in j.find_all('div',class_='stars')]

给予：

['1.0 star rating', '1.0 star rating', '3.0 star rating', '1.0 star rating', '1.0 star rating', '1.0 star rating', '1.0 star rating', '1.0 star rating', '1.0 star rating', '1.0 star rating', '5.0 star rating', '1.0 star rating', '1.0 star rating', '1.0 star rating', '1.0 star rating', '1.0 star rating', '1.0 star rating', '5.0 star rating', '2.0 star rating', '5.0 star rating', '1.0 star rating', '2.0 star rating', '1.0 star rating', '5.0 star rating', '1.0 star rating', '5.0 star rating']

Answer 2

下面怎么样

from bs4 import BeautifulSoup

html = """<div class="stars" title="4.0 star rating">
<i class="star star--large star-0"></i><i class="star star--large star- 
1"></i><i class="star star--large star-2"></i><i class="star star--large 
star-3"></i><i class="star star--large star-4 star--large--muted"></i> 
</div>"""

soup = BeautifulSoup(html, features="lxml")
element = soup.select('.stars')[0]['title']
print(element)

---

打印

4.0 star rating

---

使用网址

import requests
from bs4 import BeautifulSoup

url = 'https://www.sitejabber.com/reviews/apple.com?page={n}&sort=Reviews.processed&direction=DESC#reviews'
page = requests.get(url=url)

soup = BeautifulSoup(page.text, features="lxml")

elements = soup.select('.stars')
# print(elements)

for element in elements:
    print(element['title'])

---

打印

4.0 star rating
3.8 star rating
3.7 star rating
4.3 star rating
3.8 star rating
4.2 star rating
0.0 star rating
0.0 star rating
5.0 star rating
1.0 star rating
1.0 star rating
1.0 star rating
3.0 star rating
1.0 star rating
1.0 star rating
1.0 star rating
1.0 star rating
1.0 star rating
1.0 star rating
1.0 star rating
5.0 star rating
1.0 star rating
1.0 star rating
1.0 star rating
1.0 star rating
1.0 star rating
1.0 star rating
5.0 star rating
2.0 star rating
5.0 star rating
1.0 star rating
2.0 star rating
1.0 star rating
5.0 star rating
1.0 star rating
5.0 star rating
4.3 star rating
3.5 star rating
4.7 star rating
3.7 star rating
4.8 star rating
3.7 star rating