如何在请求中使用多线程？答案

【问题标题】：How can I use multithreading with requests?如何在请求中使用多线程？
【发布时间】：2019-08-02 17:56:42
【问题描述】：

您好，我有这段代码使用 Python，它使用 requests 模块：

import requests

url1 = "myurl1" # I do not remember exactly the exact url
reponse1 = requests.get(url1)
temperature1 = reponse1.json()["temperature"]

url2 = "myurl2" # I do not remember exactly the exact url
reponse2 = requests.get(url2)
temperature2 = reponse2.json()["temp"]

url3 = "myurl3" # I do not remember exactly the exact url
reponse3 = requests.get(url3)
temperature3 = reponse3.json()[0]

print(temperature1)
print(temperature2)
print(temperature3)

实际上我不得不告诉你这有点慢......你有解决方案来提高我的代码速度吗？我想用多线程，但我不知道如何使用它......

非常感谢！

【问题讨论】：

我想如果你做一些搜索，这里有关于 SO 的问答，其中有使用多处理、线程、异步或 cuncurrent.futures 运行 Web 请求的示例。
相关：What is the fastest way to send 100,000 HTTP requests in Python? ... A very simple multithreading parallel URL fetching (without queue) ... 等等。
concurrent.futures 文档甚至有 an example
The Requests documentation 指向其他解决方案。
A very simple multithreading parallel URL fetching (without queue)的可能重复

标签： python python-3.x multithreading python-2.7 python-requests

【解决方案1】：

试试Python executors:

import requests
from concurrent.futures import ThreadPoolExecutor, as_completed
from multiprocessing import cpu_count

urls = ['/url1', '/url2', '/url3']
with ThreadPoolExecutor(max_workers=2*cpu_count()) as executor:
    future_to_url = {executor.submit(requests.get, url): url for url in urls}
    for future in as_completed(future_to_url):
        response = future.result()  # TODO: handle exceptions here
        url = future_to_url[future]
        # TODO: do something with that data

【讨论】：

concurrent.futures 让一切变得简单！