通过 Selenium 和 Java 使用 GeckoDriver 和 Firefox 无法点击链接

Question

尝试通过 Selenium 打开链接时出现错误：

• FireFox V.73.0（64 位）
• Geckodriver 驱动程序版本-v0.26。 0

帮我解决问题

public By searchResultLinktLocator(int searchResult) {
    By searchResultLinktLocator = By.xpath("//li[@data-result-number='" + searchResult + "']" + "//a");
    return searchResultLinktLocator;
}

public BBCSearchPage openSearchResultLink(int searchResult) {
    actions.click(driver.findElement(searchResultLinktLocator(searchResult))).perform();
    return this; 
}

此功能的链接打不开：

searchPage.openSearchResultLink(1);

打不开链接的页面：https://www.bbc.co.uk/search?q=Andersen

1581606071797   mozrunner::runner   INFO    Running command: "C:\\Program Files\\Mozilla Firefox\\firefox.exe" "-marionette" "-foreground" "-no-remote" "-profile" "C:\\Users\\anduser\\AppData\\Local\\Temp\\rust_mozprofileZpIGZz"
1581606072297   addons.webextension.doh-rollout@mozilla.org WARN    Loading extension 'doh-rollout@mozilla.org': Reading manifest: Invalid extension permission: networkStatus
1581606072791   addons.webextension.screenshots@mozilla.org WARN    Loading extension 'screenshots@mozilla.org': Reading manifest: Invalid extension permission: mozillaAddons
1581606072791   addons.webextension.screenshots@mozilla.org WARN    Loading extension 'screenshots@mozilla.org': Reading manifest: Invalid extension permission: telemetry
1581606072791   addons.webextension.screenshots@mozilla.org WARN    Loading extension 'screenshots@mozilla.org': Reading manifest: Invalid extension permission: resource://pdf.js/
1581606072791   addons.webextension.screenshots@mozilla.org WARN    Loading extension 'screenshots@mozilla.org': Reading manifest: Invalid extension permission: about:reader*
JavaScript error: resource://gre/modules/XULStore.jsm, line 66: Error: Can't find profile directory.
1581606075330   Marionette  INFO    Listening on port 53959
1581606075570   Marionette  WARN    TLS certificate errors will be ignored for this session
фев 13, 2020 6:01:15 PM org.openqa.selenium.remote.ProtocolHandshake createSession
INFO: Detected dialect: W3C
JavaScript error: https://nav.files.bbci.co.uk/orbit/1b2e292884201dd13064a9204e177864/js/require.min.js, line 1: TypeError: e.slice is not a function
1581606081548   Marionette  WARN    TimedPromise timed out after 0 ms: stacktrace:
TimedPromise/<@chrome://marionette/content/sync.js:244:13
TimedPromise@chrome://marionette/content/sync.js:229:10
Sleep@chrome://marionette/content/sync.js:282:10
dispatchPause@chrome://marionette/content/action.js:1482:10
toEvents/<@chrome://marionette/content/action.js:1156:16
action.dispatchTickActions@chrome://marionette/content/action.js:1060:35
action.dispatch/chainEvents<@chrome://marionette/content/action.js:1028:20
action.dispatch@chrome://marionette/content/action.js:1034:5
performActions@chrome://marionette/content/listener.js:850:16
dispatch/</req<@chrome://marionette/content/listener.js:527:14
dispatch/<@chrome://marionette/content/listener.js:520:15
1581606081697   Marionette  WARN    TimedPromise timed out after 0 ms: stacktrace:
TimedPromise/<@chrome://marionette/content/sync.js:244:13
TimedPromise@chrome://marionette/content/sync.js:229:10
Sleep@chrome://marionette/content/sync.js:282:10
dispatchPause@chrome://marionette/content/action.js:1482:10
toEvents/<@chrome://marionette/content/action.js:1156:16
action.dispatchTickActions@chrome://marionette/content/action.js:1060:35
action.dispatch/chainEvents<@chrome://marionette/content/action.js:1028:20
1581606081712   Marionette  WARN    TimedPromise timed out after 0 ms: stacktrace:
TimedPromise/<@chrome://marionette/content/sync.js:244:13
TimedPromise@chrome://marionette/content/sync.js:229:10
Sleep@chrome://marionette/content/sync.js:282:10
dispatchPause@chrome://marionette/content/action.js:1482:10
toEvents/<@chrome://marionette/content/action.js:1156:16
action.dispatchTickActions@chrome://marionette/content/action.js:1060:35
action.dispatch/chainEvents<@chrome://marionette/content/action.js:1028:20
1581606081730   Marionette  WARN    TimedPromise timed out after 0 ms: stacktrace:
TimedPromise/<@chrome://marionette/content/sync.js:244:13
TimedPromise@chrome://marionette/content/sync.js:229:10
Sleep@chrome://marionette/content/sync.js:282:10
dispatchPause@chrome://marionette/content/action.js:1482:10
toEvents/<@chrome://marionette/content/action.js:1156:16
action.dispatchTickActions@chrome://marionette/content/action.js:1060:35
action.dispatch/chainEvents<@chrome://marionette/content/action.js:1028:20
1581606081732   Marionette  WARN    TimedPromise timed out after 0 ms: stacktrace:
TimedPromise/<@chrome://marionette/content/sync.js:244:13
TimedPromise@chrome://marionette/content/sync.js:229:10
Sleep@chrome://marionette/content/sync.js:282:10
dispatchPause@chrome://marionette/content/action.js:1482:10
toEvents/<@chrome://marionette/content/action.js:1156:16
action.dispatchTickActions@chrome://marionette/content/action.js:1060:35
action.dispatch/chainEvents<@chrome://marionette/content/action.js:1028:20
1581606081734   Marionette  WARN    TimedPromise timed out after 0 ms: stacktrace:
TimedPromise/<@chrome://marionette/content/sync.js:244:13
TimedPromise@chrome://marionette/content/sync.js:229:10
Sleep@chrome://marionette/content/sync.js:282:10
dispatchPause@chrome://marionette/content/action.js:1482:10
toEvents/<@chrome://marionette/content/action.js:1156:16
action.dispatchTickActions@chrome://marionette/content/action.js:1060:35
action.dispatch/chainEvents<@chrome://marionette/content/action.js:1028:20
1581606081747   Marionette  WARN    TimedPromise timed out after 0 ms: stacktrace:
TimedPromise/<@chrome://marionette/content/sync.js:244:13
TimedPromise@chrome://marionette/content/sync.js:229:10
Sleep@chrome://marionette/content/sync.js:282:10
dispatchPause@chrome://marionette/content/action.js:1482:10
toEvents/<@chrome://marionette/content/action.js:1156:16
action.dispatchTickActions@chrome://marionette/content/action.js:1060:35
action.dispatch/chainEvents<@chrome://marionette/content/action.js:1028:20
1581606081752   Marionette  WARN    TimedPromise timed out after 0 ms: stacktrace:
TimedPromise/<@chrome://marionette/content/sync.js:244:13
TimedPromise@chrome://marionette/content/sync.js:229:10
Sleep@chrome://marionette/content/sync.js:282:10
dispatchPause@chrome://marionette/content/action.js:1482:10
toEvents/<@chrome://marionette/content/action.js:1156:16
action.dispatchTickActions@chrome://marionette/content/action.js:1060:35
action.dispatch/chainEvents<@chrome://marionette/content/action.js:1028:20
1581606081753   Marionette  WARN    TimedPromise timed out after 0 ms: stacktrace:
TimedPromise/<@chrome://marionette/content/sync.js:244:13
TimedPromise@chrome://marionette/content/sync.js:229:10
Sleep@chrome://marionette/content/sync.js:282:10
dispatchPause@chrome://marionette/content/action.js:1482:10
toEvents/<@chrome://marionette/content/action.js:1156:16
action.dispatchTickActions@chrome://marionette/content/action.js:1060:35
action.dispatch/chainEvents<@chrome://marionette/content/action.js:1028:20
1581606081755   Marionette  WARN    TimedPromise timed out after 0 ms: stacktrace:
TimedPromise/<@chrome://marionette/content/sync.js:244:13
TimedPromise@chrome://marionette/content/sync.js:229:10
Sleep@chrome://marionette/content/sync.js:282:10
dispatchPause@chrome://marionette/content/action.js:1482:10
toEvents/<@chrome://marionette/content/action.js:1156:16
action.dispatchTickActions@chrome://marionette/content/action.js:1060:35
action.dispatch/chainEvents<@chrome://marionette/content/action.js:1028:20
1581606081755   Marionette  WARN    TimedPromise timed out after 0 ms: stacktrace:
TimedPromise/<@chrome://marionette/content/sync.js:244:13
TimedPromise@chrome://marionette/content/sync.js:229:10
Sleep@chrome://marionette/content/sync.js:282:10
dispatchPause@chrome://marionette/content/action.js:1482:10
toEvents/<@chrome://marionette/content/action.js:1156:16
action.dispatchTickActions@chrome://marionette/content/action.js:1060:35
action.dispatch/chainEvents<@chrome://marionette/content/action.js:1028:20
1581606081863   Marionette  WARN    TimedPromise timed out after 0 ms: stacktrace:
TimedPromise/<@chrome://marionette/content/sync.js:244:13
TimedPromise@chrome://marionette/content/sync.js:229:10
Sleep@chrome://marionette/content/sync.js:282:10
dispatchPause@chrome://marionette/content/action.js:1482:10
toEvents/<@chrome://marionette/content/action.js:1156:16
action.dispatchTickActions@chrome://marionette/content/action.js:1060:35
action.dispatch/chainEvents<@chrome://marionette/content/action.js:1028:20
1581606081874   Marionette  WARN    TimedPromise timed out after 0 ms: stacktrace:
TimedPromise/<@chrome://marionette/content/sync.js:244:13
TimedPromise@chrome://marionette/content/sync.js:229:10
Sleep@chrome://marionette/content/sync.js:282:10
dispatchPause@chrome://marionette/content/action.js:1482:10
toEvents/<@chrome://marionette/content/action.js:1156:16
action.dispatchTickActions@chrome://marionette/content/action.js:1060:35
action.dispatch/chainEvents<@chrome://marionette/content/action.js:1028:20
1581606081882   Marionette  WARN    TimedPromise timed out after 0 ms: stacktrace:
TimedPromise/<@chrome://marionette/content/sync.js:244:13
TimedPromise@chrome://marionette/content/sync.js:229:10
Sleep@chrome://marionette/content/sync.js:282:10
dispatchPause@chrome://marionette/content/action.js:1482:10
toEvents/<@chrome://marionette/content/action.js:1156:16
action.dispatchTickActions@chrome://marionette/content/action.js:1060:35
action.dispatch/chainEvents<@chrome://marionette/content/action.js:1028:20
1581606081886   Marionette  WARN    TimedPromise timed out after 0 ms: stacktrace:
TimedPromise/<@chrome://marionette/content/sync.js:244:13
TimedPromise@chrome://marionette/content/sync.js:229:10
Sleep@chrome://marionette/content/sync.js:282:10
dispatchPause@chrome://marionette/content/action.js:1482:10
toEvents/<@chrome://marionette/content/action.js:1156:16
action.dispatchTickActions@chrome://marionette/content/action.js:1060:35
action.dispatch/chainEvents<@chrome://marionette/content/action.js:1028:20
1581606081888   Marionette  WARN    TimedPromise timed out after 0 ms: stacktrace:
TimedPromise/<@chrome://marionette/content/sync.js:244:13
TimedPromise@chrome://marionette/content/sync.js:229:10
Sleep@chrome://marionette/content/sync.js:282:10
dispatchPause@chrome://marionette/content/action.js:1482:10
toEvents/<@chrome://marionette/content/action.js:1156:16
action.dispatchTickActions@chrome://marionette/content/action.js:1060:35
action.dispatch/chainEvents<@chrome://marionette/content/action.js:1028:20
1581606081899   Marionette  WARN    TimedPromise timed out after 0 ms: stacktrace:
TimedPromise/<@chrome://marionette/content/sync.js:244:13
TimedPromise@chrome://marionette/content/sync.js:229:10
Sleep@chrome://marionette/content/sync.js:282:10
dispatchPause@chrome://marionette/content/action.js:1482:10
toEvents/<@chrome://marionette/content/action.js:1156:16
action.dispatchTickActions@chrome://marionette/content/action.js:1060:35
action.dispatch/chainEvents<@chrome://marionette/content/action.js:1028:20
1581606081931   Marionette  WARN    TimedPromise timed out after 0 ms: stacktrace:
TimedPromise/<@chrome://marionette/content/sync.js:244:13
TimedPromise@chrome://marionette/content/sync.js:229:10
Sleep@chrome://marionette/content/sync.js:282:10
dispatchPause@chrome://marionette/content/action.js:1482:10
toEvents/<@chrome://marionette/content/action.js:1156:16
action.dispatchTickActions@chrome://marionette/content/action.js:1060:35
action.dispatch/chainEvents<@chrome://marionette/content/action.js:1028:20
1581606081945   Marionette  WARN    TimedPromise timed out after 0 ms: stacktrace:
TimedPromise/<@chrome://marionette/content/sync.js:244:13
TimedPromise@chrome://marionette/content/sync.js:229:10
Sleep@chrome://marionette/content/sync.js:282:10
dispatchPause@chrome://marionette/content/action.js:1482:10
toEvents/<@chrome://marionette/content/action.js:1156:16
action.dispatchTickActions@chrome://marionette/content/action.js:1060:35
action.dispatch/chainEvents<@chrome://marionette/content/action.js:1028:20
JavaScript error: , line 0: uncaught exception: undefined

Answer 1

我采用了您的代码并简化了结构，并以最少的代码行运行了测试，如下所示：

• 代码块：

  public class A_Firefox_Test 
  {
      public static void main(String[] args) 
      {
          System.setProperty("webdriver.gecko.driver", "C:/Utility/BrowserDrivers/geckodriver.exe");
          WebDriver driver = new FirefoxDriver();
          driver.get("https://www.bbc.co.uk/search?q=Andersen");
          int searchResult = 1;
          new WebDriverWait(driver, 10).until(ExpectedConditions.elementToBeClickable(By.xpath("//li[@data-result-number='" + searchResult + "']"))).click();
      }
  }
  

与您的观察类似，我遇到了同样的障碍，即所需的链接不可点击，另外还有一个 警告：

[Parent 7472, Gecko_IOThread] WARNING: file z:/task_1579290903/build/src/ipc/chromium/src/base/process_util_win.cc, line 160

这意味着服务在process_util_win.cc 内被中断，即进程 模块。

---

深潜

我继续检查 webpage 的 DOM Tree 时发现某些 <script> 标记指的是具有关键字 dist 的 JavaScripts 。举个例子：

• map['idcta-v2/idcta-1'] = 'https://static.files.bbci.co.uk/account/id-cta/1.36.1/modules/idcta-v2/dist/idcta-1.min';
• document.write('<script type="text/javascript" src="https://static.bbc.co.uk/bbcdotcom/3.6.969/script/dist/bbcdotcom.dev.js">\x3C/script>');
• document.write('<script type="text/javascript" src="https://static.bbc.co.uk/bbcdotcom/3.6.969/script/dist/bbcdotcom.js">\x3C/script>');

这清楚地表明该网站受到 Bot Management 服务提供商 Distil Networks 的保护，并且 ChromeDriver 的导航被检测到并随后被阻止强>。

---

蒸馏

根据文章There Really Is Something About Distil.it...：

Distil 通过观察网站行为和识别抓取工具特有的模式来保护网站免受自动内容抓取机器人的攻击。当 Distil 在一个站点上识别出恶意机器人时，它会创建一个列入黑名单的行为配置文件，并部署到其所有客户。类似于 bot 防火墙，Distil 检测模式并做出反应。

进一步，

"One pattern with Selenium was automating the theft of Web content"，Distil 首席执行官 Rami Essaid 在上周接受采访时表示。 "Even though they can create new bots, we figured out a way to identify Selenium the a tool they're using, so we're blocking Selenium no matter how many times they iterate on that bot. We're doing that now with Python and a lot of different technologies. Once we see a pattern emerge from one type of bot, then we work to reverse engineer the technology they use and identify it as malicious".

---

参考

您可以在以下位置找到一些相关的详细讨论：

• Chrome browser initiated through ChromeDriver gets detected
• Selenium webdriver: Modifying navigator.webdriver flag to prevent selenium detection
• Akamai Bot Manager detects WebDriver driven Chrome Browsing Context
• Is there a version of selenium webdriver that is not detectable?

Answer 2

您可能在.perform() 之前缺少.build()。

actions.click(driver.findElement(searchResultLinktLocator(searchResult))).build().perform()

这是否仅在某些搜索上失败，但在其他搜索上有效？您也没有包括实例化actions 的位置，这可能有助于阐明问题。如果你更新你的帖子，我会更新我的答案。