【问题标题】:celery worker only imports tasks when not detachedcelery worker 仅在未分离时导入任务
【发布时间】:2021-02-08 09:06:05
【问题描述】:

我正在尝试让我的 django 应用程序将任务提交给 celery 工作人员,并且当工作人员附加运行时它会成功。一旦我添加了 --detach 任务就无法注册。

[2020-10-26 04:09:33,159: ERROR/MainProcess] Received unregistered task of type 'devapp.tasks.create_random_files'.
The message has been ignored and discarded.

Did you remember to import the module containing this task?
Or maybe you're using relative imports?

Please see
http://docs.celeryq.org/en/latest/internals/protocol.html
for more information.

The full contents of the message body was:
'[[20, "blah", 5120], {}, {"callbacks": null, "errbacks": null, "chain": null, "chord": null}]' (93b)
Traceback (most recent call last):
  File "/pysite/project/venv/lib/python3.6/site-packages/celery/worker/consumer/consumer.py", line 562, in on_task_received
    strategy = strategies[type_]
KeyError: 'devapp.tasks.create_random_files'

在我的 tasks.py 我有

import os
import string
import subprocess
from celery import shared_task, task

@shared_task(name='devapp.tasks.create_random_files')
def create_random_files(total,filename,size):
    for i in range(int(total)):
        filenum = str(i).zfill(5)
        rnd_filename = '/brdis/{}-{}'.format(filenum,filename)
        with open(rnd_filename, 'wb') as f:
            f.write(os.urandom(size))
            f.close
    return '{} random files created'.format(total)

在我的views.py中我有

from django.shortcuts import render
from django.http import HttpResponse

from django.contrib import messages
from django.views.generic import TemplateView
from django.views.generic.list import ListView
from django.views.generic.edit import FormView
from django.shortcuts import redirect

from .forms import CreateRandomFilesForm, GetFileListForm, ClamScanFileForm
from .tasks import create_random_files, clamscanfile

class ClamScanFileView(FormView):
    template_name = 'devapp/clamscan_file.html'
    form_class = ClamScanFileForm

    def form_valid(self, form):
        filename = form.cleaned_data.get('filename')
        clamscanfile.delay(filename)
        messages.success(self.request, 'We are scanning your file!s Wait a moment and refresh this page.')
        return redirect('files_list')

我知道它可能是相对导入(我需要其他一些东西),但我不明白为什么将 --detach 添加到 celery worker 命令会产生此错误:

celery -A project worker -E --loglevel=INFO --logfile=/var/log/celery/celeryd.log --pidfile=/var/run/celery/celeryd.pid

工作正常,但是...

celery -A project worker -E --loglevel=INFO --logfile=/var/log/celery/celeryd.log --pidfile=/var/run/celery/celeryd.pid --detach

启动worker但不注册任务?

任何帮助表示赞赏。

FWIW

celery report

software -> celery:4.4.7 (cliffs) kombu:4.6.11 py:3.6.8
            billiard:3.6.3.0 py-amqp:2.6.1
platform -> system:Linux arch:64bit, ELF
            kernel version:3.10.0-1127.19.1.el7.x86_64 imp:CPython
loader   -> celery.loaders.default.Loader
settings -> transport:amqp results:disabled

【问题讨论】:

  • 你有没有得到这个?使用 --detach 或 multi 遇到同样的问题
  • worker 将响应“inspect”命令,这意味着它连接到同一个后端,因此它确实掌握了正确的配置...
  • 尝试全局安装应用程序包,但行为仍然相同。现在没有想法了。

标签: django rabbitmq celery django-celery


【解决方案1】:

这是 4.4.7 的一个错误。降级到 4.4.6 或升级到 5 应该可以解决它。

https://github.com/celery/celery/issues/6370

【讨论】:

  • 感谢@Giannis,这是有道理的。这对你有用吗?
  • 我已经升级到 5.0.2,但是用 4.4.6 测试过,它也能正常工作。
  • 我很抱歉延迟(非常)回复。离开这个项目有一段时间了。由于与 django-celery-results 和花的冲突,目前仍无法更新。我可能不得不落花......
猜你喜欢
  • 1970-01-01
  • 2023-03-07
  • 2022-06-17
  • 2012-06-06
  • 1970-01-01
  • 2012-08-26
  • 2021-01-24
  • 2017-10-11
  • 2020-01-03
相关资源
最近更新 更多