【发布时间】：2018-07-09 08:20:45
【问题描述】：
我有一个用autoretry_for 装饰的 celery 任务，这样在发生已知异常时，它将重试该任务。这里有一个虚拟版本：
class ExpectedException(Exception):
pass
@app.task(autoretry_for=(ExpectedException,), retry_kwargs={'max_retries': 2, 'countdown': 1})
def decorated_autoretry():
logging.info(
"Attempt: {attempt} of {attempts}".format(
attempt=decorated_autoretry.request.retries, attempts=decorated_autoretry.max_retries
)
)
raise ExpectedException
运行时给出以下输出：
[2018-01-30 12:17:31,899: INFO/MainProcess] Received task: interpretationAPI.tasks.util_tasks.decorated_autoretry[9e00f56d-fb90-46db-a735-678fd0b4cb5a]
[2018-01-30 12:17:31,900: INFO/ForkPoolWorker-1] Attempt: 1 of 3
[2018-01-30 12:17:31,915: INFO/ForkPoolWorker-1] Task interpretationAPI.tasks.util_tasks.decorated_autoretry[9e00f56d-fb90-46db-a735-678fd0b4cb5a] retry: Retry in 1s: ExpectedException()
[2018-01-30 12:17:31,915: INFO/MainProcess] Received task: interpretationAPI.tasks.util_tasks.decorated_autoretry[9e00f56d-fb90-46db-a735-678fd0b4cb5a] ETA:[2018-01-30 12:17:32.901955+00:00]
[2018-01-30 12:17:33,024: INFO/ForkPoolWorker-2] Attempt: 2 of 3
[2018-01-30 12:17:33,072: INFO/MainProcess] Received task: interpretationAPI.tasks.util_tasks.decorated_autoretry[9e00f56d-fb90-46db-a735-678fd0b4cb5a] ETA:[2018-01-30 12:17:34.029462+00:00]
[2018-01-30 12:17:33,072: INFO/ForkPoolWorker-2] Task interpretationAPI.tasks.util_tasks.decorated_autoretry[9e00f56d-fb90-46db-a735-678fd0b4cb5a] retry: Retry in 1s: ExpectedException()
[2018-01-30 12:17:34,033: INFO/ForkPoolWorker-1] Attempt: 3 of 3
[2018-01-30 12:17:34,037: ERROR/ForkPoolWorker-1] Task interpretationAPI.tasks.util_tasks.decorated_autoretry[9e00f56d-fb90-46db-a735-678fd0b4cb5a] raised unexpected: ExpectedException()
Traceback (most recent call last):
File "/home/greg/gel/interpretation/.env/local/lib/python2.7/site-packages/celery/app/trace.py", line 374, in trace_task
R = retval = fun(*args, **kwargs)
File "/home/greg/gel/interpretation/.env/local/lib/python2.7/site-packages/celery/app/trace.py", line 629, in __protected_call__
return self.run(*args, **kwargs)
File "/home/greg/gel/interpretation/.env/local/lib/python2.7/site-packages/celery/app/base.py", line 474, in run
raise task.retry(exc=exc, **retry_kwargs)
File "/home/greg/gel/interpretation/.env/local/lib/python2.7/site-packages/celery/app/task.py", line 669, in retry
raise_with_context(exc)
File "/home/greg/gel/interpretation/.env/local/lib/python2.7/site-packages/celery/app/base.py", line 472, in run
return task._orig_run(*args, **kwargs)
File "/home/greg/gel/interpretation/interpretationAPI/tasks/util_tasks.py", line 25, in decorated_autoretry
raise ExpectedException
ExpectedException
在最后一次尝试中，ExpectedException 被提出，但 Celery 认为它出乎意料。
我可以在这里显式处理异常：
@app.task
def explicit_autoretry():
logging.info(
"Attempt: {attempt} of {attempts}".format(
attempt=explicit_autoretry.request.retries+1, attempts=explicit_autoretry.max_retries
)
)
try:
raise ExpectedException
except ExpectedException as e:
logging.info(msg="Received exception of type: {e_type}".format(e_type=type(e)))
try:
explicit_autoretry.retry(countdown=1)
except MaxRetriesExceededError as e:
logging.info(msg="Received exception of type: {e_type}".format(e_type=type(e)))
运行时会给出以下输出：
[2018-01-30 12:19:45,284: INFO/MainProcess] Received task: interpretationAPI.tasks.util_tasks.explicit_autoretry[b1f29107-ac73-401d-b0c3-636e91661ee9]
[2018-01-30 12:19:45,287: INFO/ForkPoolWorker-3] Attempt: 1 of 3
[2018-01-30 12:19:45,288: INFO/ForkPoolWorker-3] Received exception of type: <class 'interpretationAPI.tasks.util_tasks.ExpectedException'>
[2018-01-30 12:19:45,301: INFO/ForkPoolWorker-3] Task interpretationAPI.tasks.util_tasks.explicit_autoretry[b1f29107-ac73-401d-b0c3-636e91661ee9] retry: Retry in 1s
[2018-01-30 12:19:45,301: INFO/MainProcess] Received task: interpretationAPI.tasks.util_tasks.explicit_autoretry[b1f29107-ac73-401d-b0c3-636e91661ee9] ETA:[2018-01-30 12:19:46.288992+00:00]
[2018-01-30 12:19:47,790: INFO/ForkPoolWorker-1] Attempt: 2 of 3
[2018-01-30 12:19:47,792: INFO/ForkPoolWorker-1] Received exception of type: <class 'interpretationAPI.tasks.util_tasks.ExpectedException'>
[2018-01-30 12:19:47,839: INFO/ForkPoolWorker-1] Task interpretationAPI.tasks.util_tasks.explicit_autoretry[b1f29107-ac73-401d-b0c3-636e91661ee9] retry: Retry in 1s
[2018-01-30 12:19:47,839: INFO/MainProcess] Received task: interpretationAPI.tasks.util_tasks.explicit_autoretry[b1f29107-ac73-401d-b0c3-636e91661ee9] ETA:[2018-01-30 12:19:48.796492+00:00]
[2018-01-30 12:19:49,789: INFO/ForkPoolWorker-3] Attempt: 3 of 3
[2018-01-30 12:19:49,789: INFO/ForkPoolWorker-3] Received exception of type: <class 'interpretationAPI.tasks.util_tasks.ExpectedException'>
[2018-01-30 12:19:49,791: INFO/ForkPoolWorker-3] Task interpretationAPI.tasks.util_tasks.explicit_autoretry[b1f29107-ac73-401d-b0c3-636e91661ee9] retry: Retry in 1s
[2018-01-30 12:19:49,791: INFO/MainProcess] Received task: interpretationAPI.tasks.util_tasks.explicit_autoretry[b1f29107-ac73-401d-b0c3-636e91661ee9] ETA:[2018-01-30 12:19:50.790244+00:00]
[2018-01-30 12:19:51,791: INFO/ForkPoolWorker-1] Attempt: 4 of 3
[2018-01-30 12:19:51,791: INFO/ForkPoolWorker-1] Received exception of type: <class 'interpretationAPI.tasks.util_tasks.ExpectedException'>
[2018-01-30 12:19:51,791: INFO/ForkPoolWorker-1] Received exception of type: <class 'celery.exceptions.MaxRetriesExceededError'>
[2018-01-30 12:19:51,791: INFO/ForkPoolWorker-1] Task interpretationAPI.tasks.util_tasks.explicit_autoretry[b1f29107-ac73-401d-b0c3-636e91661ee9] succeeded in 0.00055760199939s: None
正确处理ExpectedException 和最终MaxRetriesExceededError 的每个实例，允许优雅的错误处理。
在使用autoretry_for 装饰器时，有没有办法进行这种错误处理，从而最大限度地减少显式错误处理的数量？我也尝试过使用 on_failure 处理程序，但无济于事。
【问题讨论】：
标签： python rabbitmq celery message-queue