Selenium：在动态加载网页中滚动到页面末尾

Question

我的网页在向下滚动页面时会不断加载新项目，直到所有项目都加载完毕。

我正在使用 Java 中的 Selenium，需要向下滚动到页面底部才能加载所有内容。

我尝试了几种不同的选项，比如滚动到页面底部的某个元素：

WebElement copyrightAtEndOfPage = webDriver.findElement(By.xpath("//a[@href='/utils/copyright.html']"));
((JavascriptExecutor) webDriver).executeScript("arguments[0].scrollIntoView();", copyrightAtEndOfPage);

这只是向下滚动一次，然后网页继续加载。

我也尝试了this 方法，它也只向下滚动一次，因为它只考虑浏览器高度。

非常感谢任何帮助。

Answer 1

我将为此提供 Python 代码。我认为翻译成 Java 很容易：

def scroll_down(self):
    """A method for scrolling the page."""

    # Get scroll height.
    last_height = self.driver.execute_script("return document.body.scrollHeight")

    while True:

        # Scroll down to the bottom.
        self.driver.execute_script("window.scrollTo(0, document.body.scrollHeight);")

        # Wait to load the page.
        time.sleep(2)

        # Calculate new scroll height and compare with last scroll height.
        new_height = self.driver.execute_script("return document.body.scrollHeight")

        if new_height == last_height:

            break

        last_height = new_height

希望对你有帮助！

Answer 2

感谢 Ratmir Asanov（请参阅上面批准的答案），我将 Python 代码翻译成 Java，以便其他人更容易实现。

try {
    long lastHeight = (long) ((JavascriptExecutor) webDriver).executeScript("return document.body.scrollHeight");

    while (true) {
        ((JavascriptExecutor) webDriver).executeScript("window.scrollTo(0, document.body.scrollHeight);");
        Thread.sleep(2000);

        long newHeight = (long) ((JavascriptExecutor) webDriver).executeScript("return document.body.scrollHeight");
        if (newHeight == lastHeight) {
            break;
        }
        lastHeight = newHeight;
    }
} catch (InterruptedException e) {
    e.printStackTrace();
}

Answer 3

稍微更新了 Johannes 代码以使其正常运行。

JavascriptExecutor js = (JavascriptExecutor) driver;
try {
    long lastHeight=((Number)js.executeScript("return document.body.scrollHeight")).longValue();
    while (true) {
        ((JavascriptExecutor) driver).executeScript("window.scrollTo(0, document.body.scrollHeight);");
        Thread.sleep(2000);

        long newHeight = ((Number)js.executeScript("return document.body.scrollHeight")).longValue();
        if (newHeight == lastHeight) {
            break;
        }
        lastHeight = newHeight;
    }
} catch (InterruptedException e) {
    e.printStackTrace();
}

Answer 4

Prabhat 进一步更新上述解决方案，因为它仍然给我编译错误。

    try {
        Object lastHeight = ((JavascriptExecutor) driver).executeScript("return document.body.scrollHeight");

        while (true) {
            ((JavascriptExecutor) driver).executeScript("window.scrollTo(0, document.body.scrollHeight);");
            Thread.sleep(2000);

            Object newHeight = ((JavascriptExecutor) driver).executeScript("return document.body.scrollHeight");
            if (newHeight.equals(lastHeight)) {
                break;
            }
            lastHeight = newHeight;
        }
    } catch (InterruptedException e) {
        e.printStackTrace();
    }
}

Answer 5

我找到了另一个动态加载页面的解决方案。

计算每次滚动前后显示的元素，并比较它们以确定您是否已滚动到底部。

var reachedEnd = false;
oldCount = driver.FindElements(By.CssSelector(".searchDataContainer.table-row.raw")).Count;

while (!reachedEnd)
{
    driver.FindElement(By.CssSelector("body")).SendKeys(Keys.End);
    Thread.Sleep(500);
    oldCount = driver.FindElements(By.CssSelector(".searchDataContainer.table-row.raw")).Count;

    if (newCount == oldCount)
    {
        reachedEnd = true;
    }
    else
    {
        newCount = oldCount;
    }
}

Answer 6

更新了对我有用的代码：

try {
                    long lastHeight = (long) ((JavascriptExecutor) driver).executeScript("return document.body.scrollHeight");
                    int cont=1000;
                    while (true) {
                        ((JavascriptExecutor) driver).executeScript("window.scrollTo(0, "+cont+");");
                        Thread.sleep(2000);

                        long newHeight = (long) ((JavascriptExecutor) driver).executeScript("return document.body.scrollHeight");
                        if (newHeight <= cont) {
                            break;
                        }
//                      lastHeight = newHeight;
                        cont+=500;
                    }
                } catch (InterruptedException e) {
                    e.printStackTrace();
                }