Python 多处理 Process() 在 Windows 10 上不起作用（即使是酸洗简单的字符串）

Question

我编写了一个 Python 类来并行绘制 pylot。它在 Linux 上运行良好，但是当我在 Windows 上尝试时遇到了问题。它甚至到了我不能腌制一根绳子的地步。我总是最终得到这个错误：

Traceback (most recent call last):
  File "test.py", line 50, in <module>
    test()
  File "test.py", line 7, in test
    asyncPlotter.saveLinePlotVec3("test")
  File "test.py", line 41, in saveLinePlotVec3
    args=(test, ))
  File "test.py", line 34, in process
    p.start()
  File "C:\Users\adrian\AppData\Local\Programs\Python\Python37\lib\multiprocessing\process.py", line 112, in start
    self._popen = self._Popen(self)
  File "C:\Users\adrian\AppData\Local\Programs\Python\Python37\lib\multiprocessing\context.py", line 223, in _Popen
    return _default_context.get_context().Process._Popen(process_obj)
  File "C:\Users\adrian\AppData\Local\Programs\Python\Python37\lib\multiprocessing\context.py", line 322, in _Popen
    return Popen(process_obj)
  File "C:\Users\adrian\AppData\Local\Programs\Python\Python37\lib\multiprocessing\popen_spawn_win32.py", line 89, in __init__
    reduction.dump(process_obj, to_child)
  File "C:\Users\adrian\AppData\Local\Programs\Python\Python37\lib\multiprocessing\reduction.py", line 60, in dump
    ForkingPickler(file, protocol).dump(obj)
TypeError: can't pickle weakref objects

C:\Python\MonteCarloTools>Traceback (most recent call last):
  File "<string>", line 1, in <module>
  File "C:\Users\adrian\AppData\Local\Programs\Python\Python37\lib\multiprocessing\spawn.py", line 99, in spawn_main
    new_handle = reduction.steal_handle(parent_pid, pipe_handle)
  File "C:\Users\adrian\AppData\Local\Programs\Python\Python37\lib\multiprocessing\reduction.py", line 82, in steal_handle
    _winapi.PROCESS_DUP_HANDLE, False, source_pid)
OSError: [WinError 87] The parameter is incorrect

我将其缩小到 Windows 使用的 spawn 启动方法 (https://docs.python.org/3/library/multiprocessing.html#contexts-and-start-methods)，当我在 Linux 上使用相同的启动方法时，我可以重现错误。

import multiprocessing as mp
def test():

    asyncPlotter = AsyncPlotter()

    asyncPlotter.saveLinePlotVec3("test")

    asyncPlotter.saveLinePlotVec3("test")

    asyncPlotter.saveLinePlotVec3("test")

    asyncPlotter.join()


class AsyncPlotter():

    def __init__(self, processes=mp.cpu_count()):

        self.manager = mp.Manager()
        self.nc = self.manager.Value('i', 0)
        self.pids = []
        self.processes = processes


    def linePlotVec3(self, nc, processes, test):

        self.waitOnPool(nc, processes)

        return None


    def waitOnPool(self, nc, processes):

        while nc.value >= processes:
            time.sleep(0.1)
        nc.value += 1


    def process(self, target, args):

        p = mp.Process(target=target, args=args)
        p.start()
        self.pids.append(p)


    def saveLinePlotVec3(self, test):

        self.process(target=self.linePlotVec3,
                       args=(self.nc, self.processes, test))


    def join(self):
        for p in self.pids:
            p.join()


if __name__=='__main__':
    test()

我希望我也可以在 Windows 上使用此方法腌制任何类型的字符串、列表、字典等。如果有人能找到一种在 Windows 上完成这项工作的方法，我将不胜感激。谢谢！

Answer 1

改变

self.process(target=self.linePlotVec3, args=(test, ))

到

self.process(target=AsyncPlotter.linePlotVec3, args=(test, ))

---

Process 接受类或全局方法，因此当您尝试向其传递实例方法时它会崩溃