硒没有加载页面

Question

from selenium import webdriver
driver= webdriver.Chrome()
driver.get('https://www.realestate.com.au/sold/') #opens normally but anything I search for doesn't load
driver.get('https://www.realestate.com.au/sold/in-alice+springs+-+greater+region%2c+nt%3b/list-1')

网站打开，我没有收到任何验证码或阻止，但页面没有加载

Answer 1

似乎有些“Flash”问题（您可以在“chromedriver”窗口中阅读错误：

Browser not compatible with given flash manifest

本网站使用 Flash，您的浏览器需要 Flash 播放器。 说你的浏览器是指 selenium 浏览器，而不是你手动打开的 chrome。

要在 selenium chrome 浏览器中安装“flash player”，请运行您的脚本；它将打开浏览器，然后转到https://get.adobe.com/flashplayer/ 并安装 Flash 播放器。

这应该可以解决问题

Answer 2

我猜他们正在阻止使用 selenium，看到相关请求返回 429。也许您可以通过这种方式逃脱。

Answer 3

有关您的用例的更多信息将有助于我们以更好的方式调试问题。但是使用Selenium 我能够访问该网站并检索页面源，如下所示：

• 代码块：

  from selenium import webdriver
  
  options = webdriver.ChromeOptions() 
  options.add_argument("start-maximized")
  options.add_experimental_option("excludeSwitches", ["enable-automation"])
  options.add_experimental_option('useAutomationExtension', False)
  driver = webdriver.Chrome(options=options, executable_path=r'C:\WebDrivers\chromedriver.exe')
  driver.get("https://www.realestate.com.au/sold/")
  print(driver.page_source)
  

• 控制台输出：

  <html class="js rui-burger rui-fancy-selects-on is-desktop app-loaded" lang="en"><head>
      <meta http-equiv="x-ua-compatible" content="IE=edge">
      <meta charset="utf-8">
      <meta name="viewport" content="width=1140">
      <meta name="description" content="Discover how much the latest properties have been sold for. Browse sold real estate listings to find auction results and sold prices from all over Australia.">
      .
      .
      .
  <script type="text/javascript" async="" charset="utf-8" id="utag_rea-group.sp-only_27" src="//tags.tiqcdn.com/utag/rea-group/sp-only/prod/utag.27.js?utv=ut4.44.201710110106"></script><script type="text/javascript" async="" charset="utf-8" id="utag_rea-group.sp-only_64" src="//tags.tiqcdn.com/utag/rea-group/sp-only/prod/utag.64.js?utv=ut4.44.202005070432"></script><script type="text/javascript" src="https://odb.outbrain.com/utils/get?url=http%3A%2F%2Fwww.realestate.com.au%2Fsold&amp;settings=true&amp;recs=true&amp;widgetJSId=HPR&amp;key=NANOWDGT01&amp;idx=0&amp;version=20009&amp;apv=false&amp;sig=kLvLc8wa&amp;format=html&amp;va=true&amp;et=true&amp;rand=98241&amp;pdobuid=-1&amp;osLang=en-US&amp;winW=1366&amp;winH=657&amp;scrW=1366&amp;scrH=768&amp;adblck=false&amp;secured=true&amp;cmpStat=0&amp;ccpaStat=0&amp;ref=&amp;px=119&amp;py=1959&amp;vpd=1302" charset="UTF-8" async=""></script><script src="https://safeinator.reamedia.com.au/2073b42b71554175cf729737a838c1f1c6d7f862/lib/base.min.js" async="" type="text/javascript"></script></head>
    <!--[if IE 8]>         <body class="sold rui-preload ie ie8 lt-ie9 lt-ie10" data-rui-select-auto-init="false"> <![endif]-->
    <!--[if IE 9]>         <body class="sold rui-preload ie ie9 lt-ie10" data-rui-select-auto-init="false"> <![endif]-->
    <!--[if gt IE 9]><!-->
  
    <body class="sold" data-rui-select-auto-init="false"><iframe src="https://secure-au.imrworldwide.com/storageframe.html" id="LOCSTORAGE" scrolling="no" name="empty" hidden="true" style="width: 1px; height: 1px; position: absolute; top: -7px; left: -7px; border: 0px;"></iframe>
      <!--<![endif]-->
      <div class="hero">
        <ul class="rui-visuallyhidden rui-accessibility">
      <li>
        <a href="#rui-main-nav" title="Go directly to the website navigation">skip to Main Navigation</a>
      </li>
      <li>
        <a href="#rui-main-content" title="Go directly to main content">skip to Content</a>
      </li>
        </ul>
        <header class="rui-header rui-header-inverse rui-header-not-fixed rui-clearfix" data-auto-init="false">
      <div class="rui-grid rui-header-container">
        <div class="rui-clearboth"></div>
        <button class="rui-burger-toggle rui-icon rui-icon-navdeck rui-button-no-style rui-desktop">
          <span class="rui-visuallyhidden">Menu</span>
          <span class="hit-box"></span>
        </button>
        <nav class="rui-myrea-nav">
          <div id="webCollectionsSlot"></div>
          <div id="webInboxSlot"></div>
          <div id="webInboxOverlay"></div>
          <ul class="rui-user-menu logged-out" style="">
            <li>
          <a href="/my-real-estate/login" title="Sign in">Sign in</a>
            </li>
            <li>
          <a href="/my-real-estate/register" class="rui-button-brand join" title="Create an account">Join</a>
            </li>
          </ul>
          <ul class="rui-user-menu logged-in rui-toggle-wrapper" style="display:none;">
            <li>
          <a href="#" class="rui-person-guy rui-toggle-link rui-icon rui-icon-user-off"></a>
          <div class="rui-account-menu-container">
            <ul class="rui-account-menu rui-toggle-container">
              <li class="first">
                <a href="/saved-properties" title="View saved properties">
              <span class="rui-icon rui-icon-save"></span><span>Saved properties</span>
                </a>
              </li>
              <li>
                <a href="/property/my-property" title="Track your property">
              <span class="rui-icon rui-icon-home"></span><span>Track your property</span>
                </a>
              </li>
              <li>
                <a href="https://homeloans.realestate.com.au/my-finances" title="My finances">
              <span class="rui-icon rui-icon-commission"></span><span>My finances</span>
                </a>
              </li>
              <li><a href="/my-real-estate/saved-searches-and-email-alerts" title="View property alerts">Saved searches &amp; alerts</a></li>
              <li><a href="/my-real-estate/profile" title="View profile">My profile</a></li>
              <li><a href="/my-real-estate/account" title="Manage account settings">Account settings</a>
              </li>
              <li class="last"><a href="/logout.ds" title="Log out">Log out</a></li>
            </ul>
          </div>
            </li>
          </ul>
        </nav>
      </div>
      <div class="rui-grid rui-nav-container rui-burger-container">
        <a title="realestate.com.au homepage" href="/buy" class="logo">
          <img src="//s1.rui.au.reastatic.net/rui-static/img/rea-logo-thin-white-v3.png" alt="realestate.com.au Australia lives here">
        </a>
        <nav class="rui-main-nav">
          <ul>
            <li class="rui-nav-tab buy">
          <a href="/buy" title="Real estate and property for sale">
            <span>Buy</span>
          </a>
            </li>
            .
            .
            .
            <li class="rui-nav-tab commercial">
          <a href="http://www.realcommercial.com.au" target="_blank" title="Commercial real estate for sale and lease">
            <span>Commercial</span>
          </a>
            </li>
          </ul>
        </nav>
      </div>
        </header>
  
  
        <div class="hero-content">
      <div class="search-form-container">
        <form action="listsearchview.ds" class="search-form" name="search-form">
          <h1 id="search-form-title">Search sold properties</h1>
          <div class="search-container">
            <div class="search-inner-container" style="z-index: 1;">
          <nav class="search-channel-container">
            <a href="/buy" class="rui-search-tab">Buy</a>
            <a href="/rent/" class="rui-search-tab">Rent</a>
            <a href="/sold/" class="rui-search-tab rui-search-tab-selected">Sold</a>
            <a href="/property/" class="rui-search-tab property-value-tab">Property value</a>
            <a href="/find-agent" class="rui-search-tab find-agents-tab">Find agents</a>
          </nav>
  
          <div class="search-inner-container" style="z-index: 1;">
  
  
            <div class="rui-search-container search-input-container">
              <i class="rui-icon rui-icon-search search-icon"></i>
  
              <input name="where" id="where" tabindex="2" class="rui-input rui-location-box rui-auto-complete-input" data-auto-init="false" data-location-source="consumer-suggest" data-max-results="7" data-client-source="homepage" autocomplete="off" placeholder="Search by state, suburb or postcode" data-multi-select="true" data-suggest-types="suburb,precinct,region,state,postcode" data-auto-submit="false" autofocus=""><div class="clear-text-container"><a class="rui-icon rui-icon-cross" title="Clear text"> </a></div>
              <input name="theme" id="theme" type="hidden" value="rea.sold">
              <button class="rui-search-button" tabindex="10">
                <span class="rui-visually">Search</span>
              </button>
              <div class="focus-border" style="display: none;"></div>
            </div>
          </div>
  
            </div>
            <section class="search-refinements" id="search-refinements">
  
  
          <div class="condition property-select-holder">
            <div class="select-holder">
              <label>All property types</label>
              <select class="rui-input rui-select resize-select rui-select-rendered" id="rui-property-type-select-id" name="propertyType" multiple="multiple" style="display: none;">
                <option value="All" selected="selected">All property types</option>
                <option value="House">House</option>
                <option value="unit apartment">Apartment &amp; Unit</option>
                <option value="Townhouse">Townhouse</option>
                <option value="Villa">Villa</option>
                <option value="Land">Land</option>
                <option value="Acreage">Acreage</option>
                <option value="Rural">Rural</option>
                <option value="unitblock">Block of Units</option>
                <option value="retire">Retirement Living</option>
              </select>
          </div>
          .
          .
          .
  </body></html>
  

---

更新

但是，在尝试调用 url https://www.realestate.com.au/sold/in-alice+springs+-+greater+region%2c+nt%3b/list-1' 时，似乎 Selenium 驱动的浏览器代理 正在通过指纹

检测到

• 代码块：

  driver = webdriver.Chrome(options=options, executable_path=r'C:\WebDrivers\chromedriver.exe')
  driver.get("https://www.realestate.com.au/sold/in-alice+springs+-+greater+region%2c+nt%3b/list-1'")
  

• 控制台输出：

  <html lang="en"><head>
      <meta charset="UTF-8">
      <link rel="shortcut icon" href="about:blank">
  </head>
  <body>
  <script src="/149e9513-01fa-4fb0-aad4-566afd725d1b/2d206a39-8ed7-437e-a3be-862e0f06eea3/j.js"></script>
  <script src="/149e9513-01fa-4fb0-aad4-566afd725d1b/2d206a39-8ed7-437e-a3be-862e0f06eea3/f.js"></script>
  <script src="/149e9513-01fa-4fb0-aad4-566afd725d1b/2d206a39-8ed7-437e-a3be-862e0f06eea3/fingerprint/script/kpf.js?url=/149e9513-01fa-4fb0-aad4-566afd725d1b/2d206a39-8ed7-437e-a3be-862e0f06eea3/fingerprint&amp;token=34091693-e55f-5166-d979-67e9426e1030"></script>
  
  </body></html>
  

您可以在Chrome browser initiated through ChromeDriver gets detected找到相关的详细讨论