在 python 中使用线程池时无法增加值

Question

我正在使用Python Thread Pool (Python recipe) 提供的类来模拟线程池。我正在尝试增加函数test 中的值counter。问题是它仍然存在0。我使用了Is this simple python code thread safe 中解释的lock，但仍然无法正常工作。

---

源代码

#! /usr/bin/python
# -*- coding: utf-8 -*-
from Queue import Queue
from threading import Thread
import threading

lock = threading.Lock()

class Worker(Thread):
    """Thread executing tasks from a given tasks queue"""
    def __init__(self, tasks):
        Thread.__init__(self)
        self.tasks = tasks
        self.daemon = True
        self.start()
    
    def run(self):
        while True:
            func, args, kargs = self.tasks.get()
            try: func(*args, **kargs)
            except Exception, e: print e
            self.tasks.task_done()

class ThreadPool:
    """Pool of threads consuming tasks from a queue"""
    def __init__(self, num_threads):
        self.tasks = Queue(num_threads)
        for _ in range(num_threads): Worker(self.tasks)

    def add_task(self, func, *args, **kargs):
        """Add a task to the queue"""
        self.tasks.put((func, args, kargs))

    def wait_completion(self):
        """Wait for completion of all the tasks in the queue"""
        self.tasks.join()

def exp1_thread(counter):
  with lock:
    print counter
    counter = counter + 1

def test():
  # 1) Init a Thread pool with the desired number of threads
  pool = ThreadPool(6)
  counter = 0
  for i in range(0, 10):
    pool.add_task(exp1_thread,counter)
  
   # 3) Wait for completion
  pool.wait_completion()
    
if __name__ == "__main__": 
  test()

---

输出

counter 0
counter 0
counter 0
counter 0
counter 0
counter 0
counter 0
counter 0
counter 0
counter 0

---

Answer 1

你得到全零的原因是整数值计数器是按值传递给线程的。每个线程都会收到一份计数器的副本，然后前往城镇。

您可以通过找到一种通过引用传递值的方法来解决此问题。

选项 1. 将 counter 定义为您传递的列表：

def exp1_thread(counter):
    with lock:
        print counter[0]
        counter[0] = counter[0] + 1

def test():
    # 1) Init a Thread pool with the desired number of threads
    pool = ThreadPool(6)
    counter = [0]
    for i in range(0, 10):
        pool.add_task(exp1_thread, counter)

    # 3) Wait for completion
    pool.wait_completion()

选项 2. 创建一个您传递的对象。

class Counter:
    def __init__(self, initial_count):
        self.count = initial_count

def exp1_thread(counter):
    with lock:
        print counter.count
        counter.count = counter.count + 1

def test():
    # 1) Init a Thread pool with the desired number of threads
    pool = ThreadPool(6)
    counter = Counter(0)
    for i in range(0, 10):
        pool.add_task(exp1_thread, counter)

    # 3) Wait for completion
    pool.wait_completion()

Answer 2

这与线程无关。实际原因是int 在 Python 中是不可变的。 仅增加 int 的函数不会产生预期的效果。

def inc(x):
    x +=1
y = 0
inc(y)
print y  # 0

如果要增加数字，可以将其存储在可变数据类型（例如list 或dict）中并操作列表。

Answer 3

int 与dictionaries 的工作方式不同，我相信精通 Python 逻辑的人可以解释其中的区别。这是我通常使用的逻辑并且它有效。正如我刚刚意识到的那样，这是因为您传递的是对象（字典、列表等）而不是值本身。

要么将你的变量声明为全局变量（但要小心），要么使用带有不同线程的单独键槽的字典，并在最后总结它们。

from threading import *
from time import sleep

myMap = {'counter' : 0}

class worker(Thread):
    def __init__(self, counterMap):
        Thread.__init__(self)
        self.counterMap = counterMap
        self.start()

    def run(self):
        self.counterMap['counter'] += 1

worker(myMap)
sleep(0.2)
worker(myMap)

sleep(0.2)
print(myMap)