【发布时间】：2017-01-04 19:38:22
【问题描述】：
我开始在 python 中使用 Celery 模块，并按照 Celery 网站上的说明创建了一个新脚本，但是在使用创建 Celery 工作服务器时收到错误
celery -A addTask worker --loglevel=info
addTask 是一个简单的 python 脚本返回总和：
from celery import Celery
app = Celery('addTask', broker = 'amqp://guest@localhost//')
@app.task
def add(x,y):
return x+y
另一个调用 addTask.py 的脚本是 runAddTask.py：
import addTask
if __name__=='__main__':
result = addTask.add.delay(5,5)
我使用的Celery版本是4.0.2，Python的版本是3.5.2。我在 Windows 10 上运行。我安装 rabbitmq 作为代理，所以脚本很好。这与 celeryproject 中列出的完全相同，但是，我收到如下错误消息：
C:\Users\user\desktop\parallel_python\celery> celery -A addTask worker --loglevel=info
celery : [2017-01-04 14:23:46,058: CRITICAL/MainProcess] Unrecoverable error: AttributeError("Can't pickle local object
'Pool.__init__.<locals>.Process'",)
At line:1 char:1
+ celery -A addTask worker --loglevel=info
+ ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+ CategoryInfo : NotSpecified: ([2017-01-04 14:...ls>.Process'",):String) [], RemoteException
+ FullyQualifiedErrorId : NativeCommandError
Traceback (most recent call last):
File "c:\users\user\appdata\local\programs\python\python35\lib\site-packages\celery\worker\worker.py", line 203, in start
self.blueprint.start(self)
File "c:\users\user\appdata\local\programs\python\python35\lib\site-packages\celery\bootsteps.py", line 119, in start
step.start(parent)
File "c:\users\user\appdata\local\programs\python\python35\lib\site-packages\celery\bootsteps.py", line 370, in start
return self.obj.start()
File "c:\users\user\appdata\local\programs\python\python35\lib\site-packages\celery\concurrency\base.py", line 131, in
start
self.on_start()
File "c:\users\user\appdata\local\programs\python\python35\lib\site-packages\celery\concurrency\prefork.py", line 112, in
on_start
**self.options)
File "c:\users\user\appdata\local\programs\python\python35\lib\site-packages\billiard\pool.py", line 1008, in __init__
self._create_worker_process(i)
File "c:\users\user\appdata\local\programs\python\python35\lib\site-packages\billiard\pool.py", line 1117, in
_create_worker_process
w.start()
File "c:\users\user\appdata\local\programs\python\python35\lib\site-packages\billiard\process.py", line 122, in start
self._popen = self._Popen(self)
File "c:\users\user\appdata\local\programs\python\python35\lib\site-packages\billiard\context.py", line 383, in _Popen
return Popen(process_obj)
File "c:\users\user\appdata\local\programs\python\python35\lib\site-packages\billiard\popen_spawn_win32.py", line 79, in
__init__
reduction.dump(process_obj, to_child)
File "c:\users\user\appdata\local\programs\python\python35\lib\site-packages\billiard\reduction.py", line 99, in dump
ForkingPickler(file, protocol).dump(obj)
AttributeError: Can't pickle local object 'Pool.__init__.<locals>.Process'
-------------- celery@userv4.0.2 (latentcall)
---- **** -----
--- * *** * -- Windows-10-10.0.14393-SP0 2017-01-04 14:23:46
-- * - **** ---
- ** ---------- [config]
- ** ---------- .> app: addTask:0x1d17467e0b8
- ** ---------- .> transport: amqp://guest:**@localhost:5672//
- ** ---------- .> results: disabled://
- *** --- * --- .> concurrency: 4 (prefork)
-- ******* ---- .> task events: OFF (enable -E to monitor tasks in this worker)
--- ***** -----
-------------- [queues]
.> celery exchange=celery(direct) key=celery
[tasks]
. addTask.add
Traceback (most recent call last):
File "<string>", line 1, in <module>
File "c:\users\user\appdata\local\programs\python\python35\lib\site-packages\billiard\spawn.py", line 159, in spawn_main
new_handle = steal_handle(parent_pid, pipe_handle)
File "c:\users\user\appdata\local\programs\python\python35\lib\site-packages\billiard\reduction.py", line 126, in
steal_handle
_winapi.DUPLICATE_SAME_ACCESS | _winapi.DUPLICATE_CLOSE_SOURCE)
PermissionError: [WinError 5] Access is denied
我真的被困在这里了。任何帮助是极大的赞赏！谢谢！
【问题讨论】：
-
Celery 从 4.0 开始停止支持 Windows。
-
错了，好吧。谢谢！你知道类似 Celery 的适用于 Windows 的东西吗？
-
我试过搜索，不成功。但是，Celery 团队由于缺乏资源而终止对 Windows 的支持似乎已成定局。在它自己的线程中问可能是一个很好的问题。
标签： python distributed-computing