等待元素与硒一起出现在分数网站上答案

【问题标题】：Wait element to be present with selenium on scores website等待元素与硒一起出现在分数网站上
【发布时间】：2021-05-20 07:44:20
【问题描述】：

我要刮这个页面https://www.sofascore.com/betting-tips-today 我用 selenium 创建了代码但不起作用：

from bs4 import BeautifulSoup
import time

from selenium import webdriver
from selenium.webdriver.chrome.options import Options
from selenium.webdriver.common.by import By
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.support import expected_conditions as EC

driver = webdriver.Chrome(executable_path=r"C:/chromedriver.exe") #, options=options)
url = "https://www.sofascore.com/betting-tips-today"

driver.get(url)

WebDriverWait(driver, 60).until(EC.presence_of_element_located((By.CSS_SELECTOR,'.fsFjfS'))) #wait until the element be present


time.sleep(5)
soup = BeautifulSoup(driver.page_source, 'html.parser')

它返回这个错误：

raise TimeoutException(message, screen, stacktrace) selenium.common.exceptions.TimeoutException: Message:

我尝试了没有等待但没有返回任何东西。

【问题讨论】：

你想拉什么？有一个 api，所以你可能不需要 selenium，

标签： python selenium-webdriver web-scraping

【解决方案1】：

你已经够近了。您需要为 visible 元素的 visibility_of_element_located() 诱导 WebDriverWait 而不是 presence_of_element_located()，您可以使用以下 Locator Strategy：

代码块：

driver.get("https://www.sofascore.com/betting-tips-today")
WebDriverWait(driver, 20).until(EC.visibility_of_element_located((By.CSS_SELECTOR, "div[class^='Content__PageContainer-sc-']")))
soup = BeautifulSoup(driver.page_source, 'html.parser')
print(soup)

注意：您必须添加以下导入：

from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.common.by import By
from selenium.webdriver.support import expected_conditions as EC

控制台输出：

<html data-triggered="true" lang="en">

<head>
  <meta content="width=device-width, initial-scale=1, shrink-to-fit=no" name="viewport" />
  <meta content="#0057a3" name="theme-color" />
  <link href="/static/favicon.ico" rel="icon" />
  <meta content="SofaScore.com LiveScore" name="author" />
  <meta content="app-id=1176147574/" name="apple-itunes-app" />
  <meta content="53e30a9a6b4c2d9e" name="yandex-verification" />
  <link href="https://api.sofascore.com" rel="preconnect dns-prefetch" />
  <link href="https://fonts.googleapis.com" rel="preconnect dns-prefetch" />
  <link href="https://fonts.gstatic.com" rel="preconnect dns-prefetch" />
  <link href="https://www.googletagmanager.com" rel="preconnect dns-prefetch" />
  <link href="https://securepubads.g.doubleclick.net" rel="preconnect dns-prefetch" />
  <link href="https://polyfill.io" rel="preconnect dns-prefetch" />
  <link href="/static/manifest.json" rel="manifest" />
  <link href="/static/favicon.ico" rel="shortcut icon" />
  <link href="/static/favicon.ico" rel="mask-icon" />
  <link href="/static/images/apple-icon-180x180.png" rel="apple-touch-icon-precomposed" sizes="180x180" />
  <link href="/static/images/apple-icon-152x152.png" rel="apple-touch-icon-precomposed" sizes="152x152" />
  <link href="/static/images/apple-icon-144x144.png" rel="apple-touch-icon-precomposed" sizes="144x144" />
  <link href="/static/images/apple-icon-120x120.png" rel="apple-touch-icon-precomposed" sizes="120x120" />
  <link href="/static/images/apple-icon-114x114.png" rel="apple-touch-icon-precomposed" sizes="114x114" />
  <link href="/static/images/apple-icon-76x76.png" rel="apple-touch-icon-precomposed" sizes="76x76" />
  <link href="/static/images/apple-icon-72x72.png" rel="apple-touch-icon-precomposed" sizes="72x72" />
  <link href="/static/images/apple-icon-57x57.png" rel="apple-touch-icon-precomposed" />
  <script async="" src="https://www.googletagmanager.com/gtag/js?id=G-HNQ9P9MGZR&amp;l=dataLayer&amp;cx=c" type="text/javascript"></script>
  <script async="" src="https://www.google-analytics.com/analytics.js" type="text/javascript"></script>
  <script async="" src="https://www.google-analytics.com/gtm/js?id=GTM-M8Z5Q48&amp;t=gtm5&amp;cid=640338494.1613564014&amp;aip=true" type="text/javascript"></script>
  <script async="" src="https://www.google-analytics.com/analytics.js" type="text/javascript"></script>
  <script async="" src="https://connect.facebook.net/signals/config/759407694841706?v=2.9.33&amp;r=stable"></script>
  <script async="" src="https://connect.facebook.net/en_US/fbevents.js"></script>
  <script async="" crossorigin="anonymous" src="https://connect.facebook.net/en_US/sdk.js?hash=434141e6be23d41ce5b145e37bc66356&amp;ua=modern_es6"></script>
  <script async="" src="https://www.googletagmanager.com/gtm.js?id=GTM-54Q2FDL"></script>
  <script charset="UTF-8" data-document-language="true" data-domain-script="bbfc9ddf-194b-411a-8caa-3d06175a3e3e" src="https://cookie-cdn.cookiepro.com/scripttemplates/otSDKStub.js" type="text/javascript"></script>
  <script type="text/javascript">
    function OptanonWrapper() {}
  </script>
  <script>
    window.dataLayer = window.dataLayer || [];
    (function(w, d, s, l, i) {
      w[l] = w[l] || []
      w[l].push({
        'gtm.start': new Date().getTime(),
        event: 'gtm.js'
      })
      var f = d.getElementsByTagName(s)[0],
        j = d.createElement(s),
        dl = l != 'dataLayer' ? '&l=' + l : ''
      j.async = true
      j.src = 'https://www.googletagmanager.com/gtm.js?id=' + i + dl
      f.parentNode.insertBefore(j, f)
    })(window, document, 'script', 'dataLayer', 'GTM-54Q2FDL')

    function gtag() {
      window.dataLayer.push(arguments)
    }
  </script><noscript><iframe height="0" src="https://www.googletagmanager.com/ns.html?id=GTM-54Q2FDL" style="display:none;visibility:hidden" width="0"></iframe></noscript>
  <script defer="" src="/static/scripts/isMobileDevice.js"></script>
  <script defer="" src="/static/scripts/deferredStyles.js"></script>

【讨论】：

不返回数据
@khaledkoubaa 您的主要问题是 TimeoutException 我已在此答案中解决。您没有提到任何数据，这听起来像是一个不同的要求。你能根据你的新要求提出一个新问题吗？ Stackoverflow 贡献者将很乐意为您提供帮助。