如何使用 python selenium 抓取具有相同 div 的元素的数量

Question

我正在尝试抓取该网站中的所有匹配项

https://web.bet9ja.com/Sport/OddsToday.aspx?IDSport=590

我需要的要求是

1.单击我共享的链接中的匹配名称，例如 kuttosh kujand 并抓取数据，然后再次导航并再次单击另一个匹配名称，该过程应针对链接中存在的匹配完成

到目前为止，我用我的代码编写了这样的代码，我能够完成我上面提到的匹配过程，但是我怎样才能完成所有匹配的过程

我写的代码：

# Here using selenium for scraping
# importing necessary modules
import selenium.webdriver
from selenium import webdriver
from selenium.webdriver.common.keys import Keys
from selenium.webdriver.common.by import By
import mysql.connector
import pymysql
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.support import expected_conditions as EC

# the relevant url
url = 'https://web.bet9ja.com/Sport/OddsToday.aspx?IDSport=590'

# the driver path
driver = webdriver.Chrome(r"c:/Users/SATYA/mysite/chromedriver")
driver.get(url)
driver.implicitly_wait(10) # seconds
buttons = WebDriverWait(driver,15).until(EC.presence_of_all_elements_located((By.CSS_SELECTOR, "div.Event.ng-binding")))
for btn in buttons:
    btn.click()
    headings= [item.text for item in driver.find_elements_by_css_selector("div.SECQ.ng-binding")]
    keys = [item.text for item in driver.find_elements_by_css_selector("div.SEOdd.g1")]
    values = [item.text for item in driver.find_elements_by_css_selector("div.SEOddLnk.ng-binding")]
    driver.execute_script("window.history.go(-1)")
    print(headings,keys,values)

谁能帮我解决这个问题

在抓取第一个匹配数据后，我编写的代码出现此错误

Traceback (most recent call last):
  File "dynamicscrape.py", line 21, in <module>
    btn.click()
  File "C:\Users\SATYA\AppData\Local\Programs\Python\Python37\lib\site-packages\selenium\webdriver\remote\webelement.py", line 80, in click
    self._execute(Command.CLICK_ELEMENT)
  File "C:\Users\SATYA\AppData\Local\Programs\Python\Python37\lib\site-packages\selenium\webdriver\remote\webelement.py", line 633, in _execute
    return self._parent.execute(command, params)
  File "C:\Users\SATYA\AppData\Local\Programs\Python\Python37\lib\site-packages\selenium\webdriver\remote\webdriver.py", line 321, in execute
    self.error_handler.check_response(response)
  File "C:\Users\SATYA\AppData\Local\Programs\Python\Python37\lib\site-packages\selenium\webdriver\remote\errorhandler.py", line 242, in check_response
    raise exception_class(message, screen, stacktrace)
selenium.common.exceptions.StaleElementReferenceException: Message: stale element reference: element is not attached to the page document
  (Session info: chrome=81.0.4044.113)

Answer 1

stale element reference: element is not attached to the page document 错误发生在元素未附加到您在刷新页面后已捕获的页面上。

要克服这个问题，您需要再次重新分配元素以避免陈旧。

buttons = WebDriverWait(driver,15).until(EC.presence_of_all_elements_located((By.CSS_SELECTOR, "div.Event.ng-binding")))
for btn in range(len(buttons)):
    #elements re-assigned again to avoid stale.
    buttons = WebDriverWait(driver, 15).until(EC.presence_of_all_elements_located((By.CSS_SELECTOR, "div.Event.ng-binding")))
    buttons[btn].click()
    headings= [item.text for item in driver.find_elements_by_css_selector("div.SECQ.ng-binding")]
    keys = [item.text for item in driver.find_elements_by_css_selector("div.SEOdd.g1")]
    values = [item.text for item in driver.find_elements_by_css_selector("div.SEOddLnk.ng-binding")]
    driver.execute_script("window.history.go(-1)")
    print(headings,keys,values)

---

If 子句。

buttons = WebDriverWait(driver,15).until(EC.presence_of_all_elements_located((By.CSS_SELECTOR, "div.Event.ng-binding")))
for btn in range(len(buttons)):
    buttons = WebDriverWait(driver, 15).until(EC.presence_of_all_elements_located((By.CSS_SELECTOR, "div.Event.ng-binding")))
    if (btn==1) or (btn==3) or (btn==4):
        buttons[btn].click()
        headings= [item.text for item in driver.find_elements_by_css_selector("div.SECQ.ng-binding")]
        keys = [item.text for item in driver.find_elements_by_css_selector("div.SEOdd.g1")]
        values = [item.text for item in driver.find_elements_by_css_selector("div.SEOddLnk.ng-binding")]
        driver.execute_script("window.history.go(-1)")
        print(headings,keys,values)