celery 每隔一段时间发送任务，但每隔一段时间才接收和执行答案

【问题标题】：celery sends task every period, but only receive and execute every other periodcelery 每隔一段时间发送任务，但每隔一段时间才接收和执行
【发布时间】：2016-11-24 21:07:30
【问题描述】：

我将时间段设置为 1 分钟。 Celery 确实按预期每分钟发送一次任务，但是每隔一分钟接收和执行一次任务。

[2016-11-24 15:47:48,653: INFO/MainProcess] mingle: searching for neighbors
[2016-11-24 15:47:49,669: INFO/MainProcess] mingle: sync with 1 nodes
[2016-11-24 15:47:49,670: INFO/MainProcess] mingle: sync complete
[2016-11-24 15:47:49,691: WARNING/MainProcess] celery@testTask ready.
[2016-11-24 15:48:00,011: INFO/Beat] Scheduler: Sending due task every-minute (tasks.download_reports)
[2016-11-24 15:49:00,053: INFO/Beat] Scheduler: Sending due task every-minute (tasks.download_reports)
[2016-11-24 15:49:00,057: INFO/MainProcess] Received task: tasks.download_reports[a927e0a5-8c45-4a2b-b05f-ab1e19156ada]
[2016-11-24 15:49:00,061: WARNING/Worker-17] this is a task
[2016-11-24 15:49:00,115: INFO/MainProcess] Task tasks.download_reports[a927e0a5-8c45-4a2b-b05f-ab1e19156ada] succeeded in 0.0551409143955s: None
[2016-11-24 15:50:00,053: INFO/Beat] Scheduler: Sending due task every-minute (tasks.download_reports)
[2016-11-24 15:51:00,076: INFO/Beat] Scheduler: Sending due task every-minute (tasks.download_reports)
[2016-11-24 15:51:00,079: INFO/MainProcess] Received task: tasks.download_reports[0015bc12-4452-486e-98b8-b8b61f1eb56c]
[2016-11-24 15:51:00,081: WARNING/Worker-10] this is a task
[2016-11-24 15:51:00,127: INFO/MainProcess] Task tasks.download_reports[0015bc12-4452-486e-98b8-b8b61f1eb56c] succeeded in 0.0465096402913s: None
... ...

这是我的 celeryconfig 文件：

# CELERY
BROKER_URL = 'amqp://guest@localhost//'
CELERY_RESULT_BACKEND = 'amqp'
CELERY_ACCEPT_CONTENT = ['application/json']
CELERY_TASK_SERIALIZER = 'json'
CELERY_RESULT_SERIALIZER = 'json'
CELERY_TIMEZONE = 'US/Eastern'

from celery.schedules import crontab

CELERYBEAT_SCHEDULE = {
    'every-minute': {
        'task': 'tasks.download_reports',
        'schedule': crontab(minute='*/1'),
        'args': (),
    },
}

这是任务文件：

from celery import Celery
celery = Celery('tasks')
celery.config_from_object('celeryconfig')

@celery.task
def download_reports():
  print 'this is a task'

【问题讨论】：

标签： rabbitmq celery

【解决方案1】：

我发现问题在于我同时运行两个 celery worker。所以我在消息队列中失去了 50% 的工作。

我通过使用一个 celery worker 来运行两个任务来解决这个问题。

celeryconfig 文件：

CELERYBEAT_SCHEDULE = {
    'every-minute': {
        'task': 'tasks.download_reports',
        'schedule': crontab(minute='*/1'),
        'args': (),
    },
    'every-5-minute': {
        'task': 'tasks.download_another_report',
        'schedule': crontab(minute='*/5'),
        'args': (),         
    }
}

任务文件

@celery.task
def download_reports():
  print 'this is a task'

@celery.task
def download_another_report():
  print 'this is another task'

【讨论】：