【问题标题】:How do I get list items for unordered list using selenium(python)?如何使用 selenium(python) 获取无序列表的列表项?
【发布时间】:2022-01-05 00:07:38
【问题描述】:

我正在尝试从所有这些列表项以及轻量级容器中抓取数据。我为提取列表项而编写的代码如下

driver.get('https://www.novitecgroup.com/en/brands/ferrari/roma/')
actions = ActionChains(driver)
parts_list = driver.find_element_by_class_name('tuning-parts-categories__content')
parts_list_element = parts_list.find_elements_by_tag_name('div')
temp = parts_list_element[0].find_elements_by_tag_name('li')
print(temp)

但是 temp 返回一个空列表。我也无法从羽毛灯中获取数据。提前感谢您的帮助。

1> Data within the featherlight to be extracted

2> webpage and respective source code I am trying to extract data from

3> New Error

【问题讨论】:

  • 你想从 li 标签中得到什么?
  • @ArundeepChohan 我希望从这个页面以及featherlight 中获取所有可见数据。很抱歉,我不清楚我到底在寻找什么,因为我是堆栈溢出的新手,这是我的第一个问题。下次我会非常精确。谢谢。
  • 得到了预期的输出?
  • @ArundeepChohan 不完全是,我还希望从我坚持使用的 Featherlight 中获取数据。谢谢。
  • @ArundeepChohan nope,在羽毛灯上抓取数据,例如产品描述、性能和最大扭矩,如上图 1 所示。谢谢。

标签: python selenium web-scraping


【解决方案1】:
wait=WebDriverWait(driver,60)                                 
driver.get('https://www.novitecgroup.com/en/brands/ferrari/roma/')

wait.until(EC.element_to_be_clickable((By.CSS_SELECTOR,"#cookie-confirm__submit"))).click()
parts_lists = wait.until(EC.visibility_of_all_elements_located((By.XPATH,"//div[@class='tuning-parts-categories__content']/div[1]//li/a[1]")))
for x in parts_lists:
    driver.execute_script("arguments[0].scrollIntoView();", x)
    time.sleep(2)
    driver.execute_script("arguments[0].click();", x)
    try:
        featherlight=WebDriverWait(driver,5).until(EC.visibility_of_element_located((By.CSS_SELECTOR,"div.tuning-part-popup__text")))
        product=featherlight.find_elements(By.XPATH,".//p[1]")
        if len(product) == 0:
            product = 'Empty'
        else:
            product = product[0].text
        
        box=featherlight.find_elements(By.XPATH,".//p[2]")
        if len(box) == 0:
            torque = 'Empty'
            performance = 'Empty'
        else:
            torque = box[0].text.split("\n")[1]
            performance = box[0].text.split("\n")[0]
        print(product,performance,torque)
    except Exception as e:
        print(str(e))
        pass
    close=wait.until(EC.element_to_be_clickable((By.CSS_SELECTOR,"button.featherlight-close-icon.featherlight-close")))
    driver.execute_script("arguments[0].click();", close)

进口:

from selenium.webdriver.common.by import By
from selenium.webdriver.support.ui import WebDriverWait 
from selenium.webdriver.support import expected_conditions as EC

输出:

NOVITEC N-TRONIC is a processor-controlled device to be connected to the original engine management with the Delivered Plug&Play wiring harness. The Kit includes two NOVITEC N-TRONIC modules. It is a ready to mount Plug & Play System and it is easy to install. Performance: 506 kW (688 hp) / 7.400 rpm Max. torque: 851 Nm / 3.750 rpm
NOVITEC N-TRONIC Kit, X-pipe system, catalyst-replacement pipe, SWITCHTRONIC and TECTRONIC Performance: 518 kW (704 hp) / 7.400 rpm Max. torque: 882 Nm / 3.750 rpm
NOVITEC N-TRONIC Kit, X-pipe system, sport metal catalysts, TECTRONIC and SWITCHTRONIC Performance: 514 kW (699hp) / 7.450 rpm         Max. torque: 877 Nm / 3.700 rpm
to use with NOVITEC exhaust and original exhaust system. Empty Empty
to use with NOVITEC- and original exhaust system (-4kg), 100 Zeller sport metal catalysts with reduced exhaust back pressure complete covered with high-temperature heat protection. Empty Empty
Message: 

Message: 

To use with NOVITEC sport metal catalysts or NOVITEC cat replacement pipes to avoid error indications from the exhaust system  In combination with order of a NOVITEC exhaust system Article no. F1 333 80
to use for exhaust systems with flap-regulation to open and close the flaps from the cockpit by Radio Empty Empty

【讨论】:

  • 我正在寻找所有这些 div 的数据以及当您单击其中一个时弹出的羽毛灯容器。
  • 那么像这样?
  • 我收到以下错误。请参考上图。
  • 似乎在下一行 wait.until(EC.element_to_be_clickable((By.CSS_SELECTOR, "#cookie-confirm__submit"))).click() 上再次出现错误
  • 它似乎也给我一些超时问题。
猜你喜欢
  • 1970-01-01
  • 1970-01-01
  • 2020-08-02
  • 1970-01-01
  • 1970-01-01
  • 2011-08-21
  • 1970-01-01
  • 2015-11-20
  • 1970-01-01
相关资源
最近更新 更多