无法处理下一页链接

Question

我在 python 中结合 selenium 编写了一个脚本，以从启用了 javascript 的网页中获取一些数据。在点击下一页按钮之前需要做三件事，因为下一页链接仅在网页包含搜索结果时才会出现。这三件事是：填写两个搜索框并单击搜索按钮。但是，我的脚本可以完美地完成这三件事，但是当它应该单击下一页链接时会中断（抛出超时异常）。如您所见，我已经尝试了三种不同的选项来单击下一页链接，但从未成功。我尝试使用的其余两个已被注释掉。怎样才能成功点击下一页按钮？

我尝试过的脚本：

from selenium import webdriver
from selenium.webdriver.common.keys import Keys
from selenium.webdriver.common.by import By
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.support import expected_conditions as EC

driver = webdriver.Chrome()
wait = WebDriverWait(driver, 10)
driver.get("https://brokercheck.finra.org/")

wait.until(EC.presence_of_element_located((By.CSS_SELECTOR, "[placeholder='Name or CRD#']"))).send_keys("Michael John")
wait.until(EC.presence_of_element_located((By.CSS_SELECTOR, "[placeholder='Firm Name or CRD# (optional)']"))).send_keys("Morgan Stanley")
wait.until(EC.presence_of_element_located((By.CSS_SELECTOR,'.md-button'))).click()
# wait.until(EC.element_to_be_clickable((By.CSS_SELECTOR,'.pagination-next a'))).click()
# wait.until(EC.presence_of_element_located((By.CSS_SELECTOR,'.pagination-next a'))).click()
wait.until(EC.visibility_of_element_located((By.CSS_SELECTOR,'.pagination-next a'))).click()
driver.quit()

下一页链接所在的元素：

<ul class="pagination ng-pristine ng-untouched ng-valid ng-scope ng-isolate-scope" data-ng-if="listCtrl.getTotalResults()" total-items="listCtrl.getDisplayResults()" ng-model="listCtrl.currentPage" max-size="1" page-label="listCtrl.pageLabel($page)" items-per-page="listCtrl.itemsPerPage" ng-change="listCtrl.pageChanged()" boundary-links="true" previous-text="‹" next-text="›" first-text="«" last-text="»" aria-invalid="false">
  <!-- ngIf: ::boundaryLinks --><li ng-if="::boundaryLinks" ng-class="{disabled: noPrevious()||ngDisabled}" class="pagination-first ng-scope disabled"><a href="" ng-click="selectPage(1, $event)" class="ng-binding">«</a></li><!-- end ngIf: ::boundaryLinks -->
  <!-- ngIf: ::directionLinks --><li ng-if="::directionLinks" ng-class="{disabled: noPrevious()||ngDisabled}" class="pagination-prev ng-scope disabled"><a href="" ng-click="selectPage(page - 1, $event)" class="ng-binding">‹</a></li><!-- end ngIf: ::directionLinks -->
  <!-- ngRepeat: page in pages track by $index --><li ng-repeat="page in pages track by $index" ng-class="{active: page.active,disabled: ngDisabled&amp;&amp;!page.active}" class="pagination-page ng-scope active"><a href="" ng-click="selectPage(page.number, $event)" class="ng-binding">1 of 27 pages</a></li><!-- end ngRepeat: page in pages track by $index -->
  <!-- ngIf: ::directionLinks --><li ng-if="::directionLinks" ng-class="{disabled: noNext()||ngDisabled}" class="pagination-next ng-scope"><a href="" ng-click="selectPage(page + 1, $event)" class="ng-binding">›</a></li><!-- end ngIf: ::directionLinks -->
  <!-- ngIf: ::boundaryLinks --><li ng-if="::boundaryLinks" ng-class="{disabled: noNext()||ngDisabled}" class="pagination-last ng-scope"><a href="" ng-click="selectPage(totalPages, $event)" class="ng-binding">»</a></li><!-- end ngIf: ::boundaryLinks -->
</ul>

Answer 1

页面上有两个具有相同定位器的分页：顶部和底部。

要处理顶部，您需要执行 driver.maximize_window() 使其可见，然后使用与您尝试相同的代码：

link = wait.until(EC.visibility_of_element_located((By.CSS_SELECTOR,'.pagination-next a')))
driver.execute_script('arguments[0].scrollIntoView();', link)
link.click()

处理底部分页：

wait.until(EC.visibility_of_element_located((By.XPATH,'(//*[contains(@class, "pagination-next")]//a)[2]'))).click()