【问题标题】:Multiprocessing infinite loop on windows (Python)Windows上的多处理无限循环(Python)
【发布时间】:2011-09-10 19:42:01
【问题描述】:

我正在尝试使用多个进程进行我的第一次模拟,现在每个模拟大约需要一个小时才能完成。对于那个我使用多处理导入。

当我在 for 循环中运行 Simulation() 时,所有代码都可以正常工作。当程序运行时,池中的所有工作人员都开始执行相同的任务,所以我最终得到了 4 个相同的副本,但是当他们完成后,他们会一遍又一遍地重新开始。更重要的是,当创建池时,整个程序从第 1 行同时运行四次,而不是只运行模拟。

以下代码几乎完全是多处理文档中示例的复制粘贴: http://docs.python.org/library/multiprocessing.html

我正在使用 Windows 7,我会尽快尝试在 linux 机器上运行它,但有人可以解释一下这里出了什么问题。

非常感谢

pd。 windows 7(64位) + python 2.7.1

....

code where: simulation, pulse1, pulse2, sol_ref, model, algorithm and i are defined. Also 
where the various imports are made. Also some print statements.

...


if __name__=='__main__':
    freeze_support()

    def calculate(func, args):
        result = func(*args)
        return '%s says that %s%s = %s' % (
            multiprocessing.current_process().name,
            func.__name__, args, result
            )


    PROCESSES = 4
    print 'Creating pool with %d processes\n' % PROCESSES
    pool = Pool(PROCESSES)
    print 'pool = %s' % pool
    print

    TASKS = [(simulation, (pulse1,pulse2,sol_ref,model,algorithm,i)) for i in delta_tau]

    results = [pool.apply_async(calculate, t) for t in TASKS]
    for r in results:
        print '\t', r.get()
    print

编辑:错误日志

在模拟中打印点数的行来自 if ma​​in 语句之前的代码。在这段代码中,模拟在 for 循环中运行一次,以检查它是否正常工作并给出预期的结果。

C:\Users\HP\Desktop\experiment>python amplifier_parallel.py
Number of z points:  169   dist:  20.0  mm  dz:  118.343195266  um
Number of t points:  8192.0   window span:  69.0  ps  dt:  8.4228515625  fs

delta_tau:  -6.0
delta_tau:  -3.85714285714
delta_tau:  -1.71428571429
delta_tau:  0.428571428571
delta_tau:  2.57142857143
delta_tau:  4.71428571429
delta_tau:  6.85714285714
delta_tau:  9.0
Simulation Time: 43.1258663953 seconds
Creating pool with 4 processes

pool = <multiprocessing.pool.Pool object at 0x063E66B0>

        Number of z points:  169   dist:  20.0  mm  dz:  118.343195266  um
Number of t points:  8192.0   window span:  69.0  ps  dt:  8.4228515625  fs
...3 more times...

delta_tau:  -6.0
...3more times
delta_tau:  -3.85714285714
... 3 more tiemes
delta_tau:  -1.71428571429
... 3 more tiemes
delta_tau:  0.428571428571
... 3 more tiemes
delta_tau:  2.57142857143
... 3 more tiemes3
delta_tau:  4.71428571429
... 3 more tiemes
delta_tau:  6.85714285714
... 3 more tiemes
delta_tau:  9.0
... 3 more tiemes
Simulation Time: 63.3316095785 seconds
... 3 more times with slightly different times, depending on the worker who did the job
Process PoolWorker-4:
Traceback (most recent call last):
  File "C:\Python27\lib\multiprocessing\process.py", line 232, in _bootstrap
    self.run()
  File "C:\Python27\lib\multiprocessing\process.py", line 88, in run
    self._target(*self._args, **self._kwargs)
  File "C:\Python27\lib\multiprocessing\pool.py", line 59, in worker
    task = get()
  File "C:\Python27\lib\multiprocessing\queues.py", line 352, in get
    return recv()
AttributeError: 'module' object has no attribute 'calculate'
Number of z points:  169   dist:  20.0  mm  dz:  118.343195266  um
Number of t points:  8192.0   window span:  69.0  ps  dt:  8.4228515625  fs

Process PoolWorker-3:
Traceback (most recent call last):
  File "C:\Python27\lib\multiprocessing\process.py", line 232, in _bootstrap
    self.run()
  File "C:\Python27\lib\multiprocessing\process.py", line 88, in run
    self._target(*self._args, **self._kwargs)
  File "C:\Python27\lib\multiprocessing\pool.py", line 59, in worker
    task = get()
  File "C:\Python27\lib\multiprocessing\queues.py", line 352, in get
    return recv()
AttributeError: 'module' object has no attribute 'calculate'
delta_tau:  -6.0
... this kind of stuff goes on for ever until i control-C

【问题讨论】:

  • 运行代码会打印什么?另外,当您在 get 中设置超时时会发生什么?
  • 如果定义了 pulse1,pulse2,sol_ref,model,algorithm,delta_tau,您的代码可以在 Linux 上运行。 calculate 应该由不同的工作人员调用。

标签: python windows-7 multiprocessing infinite-loop


【解决方案1】:

if __name__=="__main__" 块之外定义calculate()。子进程看不到它on Windows

【讨论】:

  • 更好但仍然无法正常工作。有了这个更改,一旦它创建了 Pool,它会在 if name__=='__main' 之前运行代码 4 次,然后从那里继续,这本身是可以接受的,但是一旦完成它就会崩溃:线程 Thread-1 中的异常(很可能在解释器关闭期间引发):回溯(最近一次调用最后一次):THX BTW:我已经用 Ubuntu 安装了一个虚拟机,原始代码在那里工作。
  • @GuillemB:使用回溯更新您的问题,或者使用重现问题的最少代码创建一个新问题。
猜你喜欢
  • 2016-12-27
  • 1970-01-01
  • 1970-01-01
  • 1970-01-01
  • 2016-03-31
  • 1970-01-01
  • 1970-01-01
  • 1970-01-01
  • 1970-01-01
相关资源
最近更新 更多