【问题标题】:Scraping reviews with selenium用 selenium 抓取评论
【发布时间】:2018-09-15 23:27:15
【问题描述】:

我正在使用 Pycharm,但它给出了错误

:selenium.common.exceptions.WebDriverException: Message: unknown error: Element <a 

我正在尝试通过转到class="quote" 中的 href 链接来删除安全评论

"

from selenium import webdriver
import pandas as pd
import re
import time

driver = webdriver.Chrome("C:\\Users\\tanmay\\Downloads\\chromedriver.exe")
driver.get("https://www.tripadvisor.in/Hotel_Review-g304551-d299120-Reviews-or10-The_Lalit_New_Delhi-New_Delhi_National_Capital_Territory_of_Delhi.html")

print("Name :",driver.find_element_by_id("HEADING").text)

driver.execute_script("window.scrollTo(0, document.body.scrollHeight);")
driver.find_element_by_xpath("//*[@id='TABS_REVIEWS']/span").click()
Element = driver.find_element_by_class_name("query_elem")
Element.send_keys("Safety")
driver.find_element_by_css_selector('.ui_icon.search.search-icon').click()
driver.execute_script("window.scrollTo(0,2050);")
driver.implicitly_wait(0.1)
time.sleep(0.1)

div = driver.find_element_by_class_name("quote")
div.find_element_by_css_selector('a').click()

【问题讨论】:

    标签: python selenium web-scraping


    【解决方案1】:
    from selenium import webdriver
    from selenium.webdriver.common.keys import Keys
    
    driver = webdriver.Chrome("#path")
    driver.get("https://www.tripadvisor.in/Hotel_Review-g304551-d299120-Reviews-or10-The_Lalit_New_Delhi-New_Delhi_National_Capital_Territory_of_Delhi.html")
    
    driver.find_element_by_class_name("query_elem").clear()
    search_bar = driver.find_element_by_class_name("query_elem")
    search_bar.send_keys('Safety')
    search_bar.send_keys(Keys.ENTER)
    
    # Below are the hrefs.
    for div in driver.find_elements_by_class_name('quote'):
        a = div.find_element_by_class_name('title')
        print(a.get_attribute('href'))
    
    # Click on the nav for next page
    # go to next page
    next_page = driver.find_element_by_class_name('next')
    next_page.click()
    

    【讨论】:

    • 不要使用 time.sleep()
    猜你喜欢
    • 2022-01-10
    • 2021-04-03
    • 2022-01-16
    • 1970-01-01
    • 2019-05-08
    • 2023-02-24
    • 2020-08-01
    • 2018-05-31
    • 1970-01-01
    相关资源
    最近更新 更多