Selenium Python一次处理多个网络驱动程序[重复]

Question

干草， 我想问如何一次处理多个 chrome 浏览器。我尝试了多个 chromedriver.exe，对于我想打开的每个浏览器，一个自己的 chromedriver.exe

driverPlace2 = "C:/Users/M/PycharmProjects/Test/chromedriver2.exe"
driverPlace3 = "C:/Users/M/PycharmProjects/Test/chromedriver3.exe"
driverPlace4 = "C:/Users/M/PycharmProjects/Test/chromedriver4.exe"
driverPlace5 = "C:/Users/M/PycharmProjects/Test/chromedriver5.exe"
driverPlace6 = "C:/Users/M/PycharmProjects/Test/chromedriver6.exe"
driverPlace7 = "C:/Users/M/PycharmProjects/Test/chromedriver7.exe"
driverPlace8 = "C:/Users/M/PycharmProjects/Test/chromedriver8.exe"

在第6个之后它开始关闭第二个，第三个......

它在控制台中什么也没显示，它刚刚开始关闭。

我该如何解决？

非常感谢您阅读和帮助我

Answer 1

您可以使用带有 Thread 的程序：（以下脚本，打开 2 个 chrome 浏览器，同时在网络上爬行）

from selenium import webdriver
from random import shuffle
from concurrent import futures
from urllib.parse import urlparse


def driver_crawl_web(first_page):
    driver = webdriver.Chrome()
    driver.get(first_page)
    while True:
        a_tags = driver.find_elements_by_xpath("//a[@href]")
        shuffle(a_tags)
        for tag in a_tags:
            href = tag.get_attribute('href')
            if check_url(href):
                selected_link = href
                break
        if selected_link == driver.current_url:
            driver.back()
            continue
        driver.get(selected_link)

def check_url(url):
    parse = urlparse(url)
    if "https" in parse.scheme:
        if "youtube" not in parse.netloc:
            return True
    return False

def crawl_with_concurrency(links):
    executor = futures.ThreadPoolExecutor(max_workers=2)
    executions = executor.map(driver_crawl_web, links)


links = ["https://www.google.com/search?q=python",
        "https://www.google.com/search?q=javascript"]

crawl_with_concurrency(links)