由于 SSL 导致的 Python 代码执行瓶颈 - 如何优化？

Question

我想提高 Python 脚本的性能，并一直在使用 cProfile 生成性能报告：

ncalls  tottime  percall  cumtime  percall filename:lineno(function)
   75   23.514    0.314   23.514    0.314 {method 'read' of '_ssl._SSLSocket' objects}
   75    8.452    0.113    8.452    0.113 {method 'do_handshake' of '_ssl._SSLSocket' objects}
   75    2.113    0.028    2.113    0.028 {method 'load_verify_locations' of '_ssl._SSLContext' objects}
   75    1.479    0.020    1.479    0.020 {method 'connect' of '_socket.socket' objects}

示例代码：

import requests
import json
from collections import defaultdict

#Added for multiprocessing
from urllib.request import urlopen
from multiprocessing.dummy import Pool as ThreadPool 

results = defaultdict(list)

# Make the Pool of workers
pool = ThreadPool(4)

# Open the urls in their own threads
# and return the results
results = pool.map(urlopen, requests.post())

  #close the pool and wait for the work to finish
pool.close()
pool.join()

for store, data in results.items():
    print('Store: {}'.format(store), end=', ')
    if data:
        for inventory in data:
            print(inventory)

Answer 1

您正在有效地测量远程网站的响应时间，这可能不是您想要的。为了最大化吞吐量（每秒发送的 HTTP 请求数或接收的数据数），您应该异步发送许多并发请求。您可以使用异步 HTTP 库，如 aiohttp 或仅使用本机 Python asyncio/asyncore。