如何使用 selenium python 向下滚动谷歌工作页面

Question

我正在尝试使用以下几行向下滚动职位发布，但有时向下滚动到最后会给出正确的结果，有时则不会。

html = driver.find_element_by_tag_name('html')
time.sleep(5)
html.send_keys(Keys.END)

谁能建议我如何向下滚动到最后，请在下面找到链接和屏幕截图。

https://www.google.com/search?q=upsc+jobs+in+india&rlz=1C1CHBF_enIN869IN869&oq=upsc+jo&aqs=chrome.1.69i57j0i433i512j0i131i433i512j0i512l3j0i131i433i512l2j0i512j0i433i512&sourceid=chrome&ie=UTF-8&ibp=htl;jobs&sa=X&sqi=2&ved=2ahUKEwjR27GN_qPzAhX4ppUCHb_0B_QQkd0GegQIORAB#fpstate=tldetail&sxsrf=AOaemvIxuJXh3if0tw7ezZfjkXRe5DSxsA:1632911697417&htivrt=jobs&htidocid=N1_BNfzt8n8auXjGAAAAAA%3D%3D

Answer 1

滚动的次数越多，获得的数据就越多，基本上它是一个动态网站。我已将50 硬编码为一个虚拟数字，您可以为此设置 100 或任何其他数字。

您可以使用示例代码：

driver = webdriver.Chrome(driver_path)
driver.maximize_window()
driver.implicitly_wait(30)
driver.get("https://www.google.com/search?q=upsc+jobs+in+india&rlz=1C1CHBF_enIN869IN869&oq=upsc+jo&aqs=chrome.1.69i57j0i433i512j0i131i433i512j0i512l3j0i131i433i512l2j0i512j0i433i512&sourceid=chrome&ie=UTF-8&ibp=htl;jobs&sa=X&sqi=2&ved=2ahUKEwjR27GN_qPzAhX4ppUCHb_0B_QQkd0GegQIORAB#fpstate=tldetail&sxsrf=AOaemvIxuJXh3if0tw7ezZfjkXRe5DSxsA:1632911697417&htivrt=jobs&htidocid=N1_BNfzt8n8auXjGAAAAAA%3D%3D")
j = 1
for i in range(50):
    element = driver.find_element(By.XPATH, f"(//div[@role='heading'])[{j}]")
    driver.execute_script("arguments[0].scrollIntoView(true);", element)
    j =  j + 1

Answer 2

你也可以试试这个滚动到最后。

driver.get("https://www.google.com/search?q=upsc+jobs+in+india&rlz=1C1CHBF_enIN869IN869&oq=upsc+jo&aqs=chrome.1.69i57j0i433i512j0i131i433i512j0i512l3j0i131i433i512l2j0i512j0i433i512&sourceid=chrome&ie=UTF-8&ibp=htl;jobs&sa=X&sqi=2&ved=2ahUKEwjR27GN_qPzAhX4ppUCHb_0B_QQkd0GegQIORAB#fpstate=tldetail&sxsrf=AOaemvIxuJXh3if0tw7ezZfjkXRe5DSxsA:1632911697417&htivrt=jobs&htidocid=N1_BNfzt8n8auXjGAAAAAA%3D%3D")
i = 0
try:
    while True:
        options = driver.find_elements_by_xpath("//div[@role='treeitem']")
        driver.execute_script("arguments[0].scrollIntoView(true);",options[i])
        i+=1
        time.sleep(.5)
except:
    pass