使用来自网页的 beautifulsoup 进行网络爬虫评分答案

【问题标题】：webscrape star rating using beautifulsoup from webpage使用来自网页的 beautifulsoup 进行网络爬虫评分
【发布时间】：2020-10-20 14:03:33
【问题描述】：

我正在尝试提取数据框中每条评论的星级以进行情绪分析。 https://www.mouthshut.com/product-reviews/Kotak-811-Mobile-Banking-reviews-925917218 这是我要抓取的网页。我对网页抓取还很陌生，所以我更喜欢beautifulsoup，因为它更容易理解。

import requests
from bs4 import BeautifulSoup
import pandas as pd

URL = ""
Final = []
for x in range(0, 8):
    if x == 1:
        URL = "https://www.mouthshut.com/product-reviews/Kotak-811-Mobile-Banking-reviews-925917218"

    else:
        URL ="https://www.mouthshut.com/product-reviews/Kotak-811-Mobile-Banking-reviews-925917218-page-{}".format(x)

    r = requests.get(URL)
    soup = BeautifulSoup(r.content, 'html.parser')
    reviews = []  # a list to store reviews

    # Use a CSS selector to extract all the review containers
    review_divs = soup.select('div.col-10.review')
    for element in review_divs :

        review = {'Review_Title': element .a.text, 'URL': element .a['href'], 'Review': element .find('div', {'class': ['more', 'reviewdata']}).text.strip()}
        reviews.append(review)

    Final.extend(reviews)

df = pd.DataFrame(Final)

非常感谢您的帮助。谢谢你

【问题讨论】：

你有什么问题？
@Jem 我想提取星级。我似乎无法弄清楚它

标签： python pandas web-scraping beautifulsoup python-requests

【解决方案1】：

您可以将以下条目添加到您的 review 字典中以获取所有给class=rating下的星星。

'Stars' : len(element.find('div', "rating").findAll("i", "rated-star"))

                                 Review_Title  ... Stars
0   Why need permission for contact,  gallery  ...     1
1        Very dull marketing for open account  ...     1
2                                  Worst bank  ...     1
3   Good interface & can be easily accessible  ...     3
4                   Best digital Bank account  ...     4
5                 Better account for everyone  ...     4
6                 Feature full Mobile banking  ...     5
7                              Very good bank  ...     4
8     Above average online banking experience  ...     3
...

【讨论】：

非常感谢，这对我有用，我也理解了你的代码行中使用的概念。