如何使用selenium循环下载多个文件

Question

我正在尝试使用 selenium 下载多个文件。该脚本适用于下载一个文件。但是在一个循环中，第一次下载后，它再也找不到任何页面元素了。

网址：https://www.grants.gov/web/grants/search-grants.html

def runExtractor(self, url):
        self.driver.get(url);
        self.driver.maximize_window();

        self.wait = WebDriverWait(self.driver, 30)

        self.driver.switch_to.frame(self.driver.find_element_by_xpath("//iframe[@id='embeddedIframe']"))

        df = pd.read_csv ('nofos.csv')
        for index, row in df.iterrows():
            try:
                time.sleep(1)
                self.search(row["oppnum"])
                time.sleep(2)
                searchGrants = self.wait.until(EC.element_to_be_clickable((By.XPATH, "//li[@id='SearchGrants-tab']/a")));
                searchGrants.click()
            except Exception:
                logging.error(traceback.format_exc())
        self.driver.close();

代码在这一行失败：

searchGrants = self.wait.until(EC.element_to_be_clickable((By.XPATH, "//li[@id='SearchGrants-tab']/a")));

这是错误：

ERROR:root:Traceback（最近一次调用最后一次）：文件 “C:\Users\name\Documents\workspace\lab\scraper\extractor.py”，行 44、在runExtractor中 searchGrants = self.wait.until(EC.element_to_be_clickable((By.XPATH, "//li[@id='SearchGrants-tab']/a")));文件 "C:\Users\name\anaconda3\envs\selenium\lib\site-packages\selenium\webdriver\support\wait.py", 第 80 行，直到 raise TimeoutException(message, screen, stacktrace) selenium.common.exceptions.TimeoutException: Message:

这是剩下的代码：

def search(self, name):
        oppNum = self.wait.until(EC.element_to_be_clickable((By.XPATH, "//*[@id='oppNum']")));
        oppNum.send_keys(name)

        searchBtn = self.wait.until(EC.element_to_be_clickable((By.XPATH, "//*[@id='searchBtn']")));
        searchBtn.click()

        archived = self.wait.until(EC.element_to_be_clickable((By.XPATH, "//*[@id='archived']")));
        archived.click()

        closed = self.wait.until(EC.element_to_be_clickable((By.XPATH, "//*[@id='closed']")));
        closed.click()

        if (self.check_exists_by_xpath("//div[@id='searchResultsDiv']/table/tbody/tr[2]/td/a")):
            selectResult = self.wait.until(EC.element_to_be_clickable((By.XPATH, "//div[@id='searchResultsDiv']/table/tbody/tr[2]/td/a")));
            selectResult.click()

            time.sleep(10)

            relatedDocumentsTab = self.wait.until(EC.element_to_be_clickable((By.XPATH, "//li[@id='relatedDocumentsTab']")));
            relatedDocumentsTab.click()

            if (self.check_exists_by_xpath("//div[@id='relatedDocumentsTable']/table/tbody/tr[3]/td[2]/a")):
                relatedDocument = self.wait.until(EC.element_to_be_clickable((By.XPATH, "//div[@id='relatedDocumentsTable']/table/tbody/tr[3]/td[2]/a")));
                relatedDocument.click()
                count = 0
                while not path.exists(self.download + "\\" + name + ".pdf" or self.download + "\\" + name + ".docx" or self.download + "\\" + name + ".doc"):
                    if count > 3:
                        logging.info("File download might be missing for " + name)
                        break
                    time.sleep(5)
                    count += 1

如何将流程返回到页面？所以我可以通过返回搜索页面来下载下一个文件。

Answer 1

由于搜索结果区域位于 iframe 内，因此我能够通过在单击搜索选项卡之前调用返回默认内容函数来解决此问题：

改变了：

for index, row in df.iterrows():
            try:
                time.sleep(1)
                self.search(row["oppnum"])
                time.sleep(2)
                searchGrants = self.wait.until(EC.element_to_be_clickable((By.XPATH, "//li[@id='SearchGrants-tab']/a")));
                searchGrants.click()

到：

for index, row in df.iterrows():
    try:
        time.sleep(1)
        self.search(row["oppnum"])
        self.driver.switch_to.default_content()
        time.sleep(1)
        searchGrants = self.wait.until(EC.element_to_be_clickable((By.XPATH, "//li[@id='SearchGrants-tab']/a")));
        searchGrants.click()
        time.sleep(1)
        self.driver.switch_to.frame(self.driver.find_element_by_xpath("//iframe[@id='embeddedIframe']"))
    except Exception:
        logging.error(traceback.format_exc())