python多线程与队列按顺序运行而不是并行答案

【问题标题】：python multithreading with queue running in sequence not in parallelpython多线程与队列按顺序运行而不是并行
【发布时间】：2018-10-23 04:24:22
【问题描述】：

为什么这段代码没有并行运行，运行它需要20秒，也就是说它是按顺序运行的。感谢您的帮助。

import time
from queue import Queue
from threading import Thread
start = time.time()
def f():
    time.sleep(0.5)
    print("yes")
    return 'yes'


def do_stuff(q):
    while True:
        output = q.get()
        q.task_done()


q = Queue(maxsize=100)
for message_nbr in range(40):
    q.put(f())

num_threads = 10

for i in range(num_threads):
    worker = Thread(target=do_stuff, args=(q, ))
    worker.setDaemon(True)
    worker.start()

q.join()
print("time: ", time.time() - start)  # ~20 seconds

【问题讨论】：

标签： python multithreading parallel-processing queue

【解决方案1】：

答案就在这里：

for message_nbr in range(40):
    q.put(f())

您将 40 个 None 实例放入队列中，因为您正在调用 f()，它返回 None 而不是传递 f（函数对象）。这个块需要 20 秒才能运行！

更改此代码

def do_stuff(q):
    while True:
        output = q.get()
        q.task_done()

到这里

def do_stuff(q):
    while True:
        output = q.get()
        output()
        q.task_done()

也是必须的（需要调用函数！）

决赛：

import time
from queue import Queue
from threading import Thread
start = time.time()
def f():
    time.sleep(0.5)
    print("yes")
    return 'yes'


def do_stuff(q):
    while True:
        output = q.get()
        output()
        q.task_done()


q = Queue(maxsize=100)
for message_nbr in range(40):
    q.put(f)

num_threads = 10

for i in range(num_threads):
    worker = Thread(target=do_stuff, args=(q, ))
    worker.setDaemon(True)
    worker.start()

q.join()
print("time: ", time.time() - start)  # time:  2.183439254760742

【讨论】：

【解决方案2】：

这个有效！

start = time.time()
def f(m):
    time.sleep(0.5)
    print("yes")
    return 'yes'


def do_stuff(q):
    while True:
        output = q.get()
        final_result = f(output)
        q.task_done()


q = Queue(maxsize=0)
for message_nbr in range(10):
#     q.put(f())
    q.put(message_nbr)

num_threads = 10

for i in range(num_threads):
    worker = Thread(target=do_stuff, args=(q, ))
    worker.setDaemon(True)
    worker.start()

q.join()
print("time: ", time.time() - start)

【讨论】：