Selenium For Loop 卡在弹出消息上

Question

我的应用程序几乎可以按预期工作。在第 5 个实例上运行循环后出现问题。搜索表明有两个结果导致相同的最终结果。发生这种情况时，我想选择两者中的第一个。

弹出消息如下所示：

我正在使用以下代码创建列表然后循环：

from selenium import webdriver
import pandas as pd
import random
from selenium.webdriver.chrome.service import Service
from selenium.webdriver.common.by import By
from selenium.webdriver.common.proxy import Proxy, ProxyType
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.support import expected_conditions as EC
import time




#service = Service('C:\Program Files\Chrome Driver\chromedriver.exe')
URL = "https://mor.nlm.nih.gov/RxClass/search?query=ALIMENTARY TRACT AND METABOLISM"
driver = webdriver.Chrome('C:\Program Files\Chrome Driver\chromedriver.exe')
driver.get(URL)

category = [my_elem.text for my_elem in WebDriverWait(driver, 20).until(EC.visibility_of_all_elements_located((By.CSS_SELECTOR, "div.drug_class img+a")))]
classid = [my_elem.text for my_elem in WebDriverWait(driver, 20).until(EC.visibility_of_all_elements_located((By.CSS_SELECTOR, "div.propText")))]
dfObj = pd.DataFrame(category) 
dfObj.columns =['Category']
dfObj.dropna(inplace = True)
new = dfObj["Category"].str.split("(", n = 1, expand = True)
dfObj["New Category"]= new[0]
dfObj["Count"]= new[1]
dfObj.drop(columns =["Category"], inplace = True)
dfObj['Count'] = dfObj['Count'].str.rstrip(')')
dfObj['IsNumber'] = dfObj['Count'].str.isnumeric()
dfObj = dfObj[(dfObj['IsNumber'] == True)]
searchcat = dfObj['New Category'].tolist()
print(searchcat)
dfObj.to_csv('tabledf.csv',index=False)
time.sleep(8)
driver.quit()

for search in searchcat:
    page = f"https://mor.nlm.nih.gov/RxClass/search?query={search}"
    driver = webdriver.Chrome('C:\Program Files\Chrome Driver\chromedriver.exe')
    driver.get(page)
    time.sleep(4)
    table = WebDriverWait(driver, 10).until(EC.presence_of_element_located((By.CSS_SELECTOR, 'tr.dbsearch')))
    time.sleep(4)
    filename = search[0:30]+'table.csv'
    pd.read_html(driver.page_source)[1].iloc[:,:-1].to_csv(filename,index=False)        
    time.sleep(4)
    driver.quit()

如果我手动单击每个搜索结果，循环将继续运行。但是，我希望 selenium 始终选择第一个选项。我该怎么办？

更新代码：

from selenium import webdriver
import pandas as pd
import random
from selenium.webdriver.chrome.service import Service
from selenium.webdriver.common.by import By
from selenium.webdriver.common.proxy import Proxy, ProxyType
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.support import expected_conditions as EC
from selenium.webdriver.support.wait import WebDriverWait, TimeoutException
import time

with webdriver.Chrome('C:\Program Files\Chrome Driver\chromedriver.exe') as driver:

    URL = "https://mor.nlm.nih.gov/RxClass/search?query=ALIMENTARY TRACT AND METABOLISM"
    driver.get(URL)
    category = [my_elem.text for my_elem in WebDriverWait(driver, 20).until(EC.visibility_of_all_elements_located((By.CSS_SELECTOR, "div.drug_class img+a")))]
    dfObj = pd.DataFrame(category) 
    dfObj.columns =["Category"]
    dfObj.dropna(inplace = True)
    new = dfObj["Category"].str.split("(", n = 1, expand = True)
    dfObj["New Category"]= new[0]
    dfObj["Count"]= new[1]
    dfObj.drop(columns =["Category"], inplace = True)
    dfObj["Count"] = dfObj["Count"].str.rstrip(')')
    dfObj["IsNumber"] = dfObj["Count"].str.isnumeric()
    dfObj = dfObj[(dfObj["IsNumber"] == True)]
    searchcat = dfObj["New Category"].tolist()
    dfObj.to_csv('tabledf.csv',index=False)
    time.sleep(3)

    for search in searchcat:
        page = f"https://mor.nlm.nih.gov/RxClass/search?query={search}"
        driver = webdriver.Chrome('C:\Program Files\Chrome Driver\chromedriver.exe')
        driver.get(page)
        table = WebDriverWait(driver, 10).until(EC.presence_of_element_located((By.CSS_SELECTOR, 'tr.dbsearch')))
        modal_wait = WebDriverWait(driver, 1)
        try:
            modal_el = modal_wait.until(EC.visibility_of_element_located((By.ID, 'optionModal')))
            modal_el.find_element(By.CSS_SELECTOR, '.uloption').click()
        except TimeoutException:
            pass
        filename = search[0:30]+'table.csv'
        classid = [my_elem.text for my_elem in WebDriverWait(driver, 20).until(EC.visibility_of_all_elements_located((By.CSS_SELECTOR, "div.table-responsive div.propText strong:nth-child(2)")))]
        classname = [my_elem.text for my_elem in WebDriverWait(driver, 20).until(EC.visibility_of_all_elements_located((By.CSS_SELECTOR, "div.table-responsive div.propText strong:nth-child(1)")))]
        classtype = [my_elem.text for my_elem in WebDriverWait(driver, 20).until(EC.visibility_of_all_elements_located((By.CSS_SELECTOR, "div.table-responsive div.propText strong:nth-child(3)")))]
        df = pd.read_html(driver.page_source)[1].iloc[:,:-1]
        df["ClassID"] = pd.Series(classid)
        df["ClassName"] = pd.Series(classname)
        df["ClassType"] = pd.Series(classtype)
        df.to_csv(filename,index=False)        
        time.sleep(4)
        driver.quit()

Answer 1

首先，我建议您使用with 上下文管理器。它将自行处理打开/关闭驱动程序（Chrome）。这样可以确保如果引发任何异常，它仍然会被关闭。

为此，请使用：

with webdriver.Chrome() as driver:
   ...

在您的代码中，我看到您为每个 URL 关闭/打开一个新浏览器。这不是必需的，不这样做会加快您的脚本速度。只需使用driver.get() 更改网址即可。

对于您的主要问题，只需添加部分代码来检测模式并选择第一个选项。类似的东西

modal_wait = WebDriverWait(driver, 1)
try:
    modal_el = modal_wait.until(EC.element_to_be_clickable((By.ID, 'optionModal')))
    modal_el.find_element(By.CSS_SELECTOR, '.uloption').click()
except TimeoutException:
    pass

您必须包含以下导入：

from selenium.webdriver.support.wait import WebDriverWait, TimeoutException
from selenium.webdriver.support import expected_conditions as EC
from selenium.webdriver.common.by import By