即使成功，长时间运行的芹菜工人任务也不会返回结果

Question

我正在运行远程 celery worker，完成一项任务大约需要 2 小时。 我使用的代理是rabbitmq，后端是rpc。完成任务后，celery worker 不会将结果返回给客户端。

Celery 版本是 4.4，操作系统是 windows

谁能告诉我可能是什么问题，因为我在 celery 日志中没有看到任何错误。

pipenv run celery worker -A src.celery_app -l info -P solo

 -------------- celery@remote_host v4.4.7 (cliffs)
--- ***** -----
-- ******* ---- Windows-10-10.0.14393-SP0 2020-09-10 20:00:18
- *** --- * ---
- ** ---------- [config]
- ** ---------- .> app:         celery_app:0x2191f95bb38
- ** ---------- .> transport:   amqp://test:**@10.1.9.159:5672/test_host
- ** ---------- .> results:     rpc://
- *** --- * --- .> concurrency: 64 (solo)
-- ******* ---- .> task events: OFF (enable -E to monitor tasks in this worker)
--- ***** -----
 -------------- [queues]
                .> celery           exchange=celery(direct) key=celery


[tasks]
  . doors.routes.abortDoors
  . doors.routes.pushDemToDoors

[2020-09-10 20:00:18,572: INFO/MainProcess] Connected to amqp://test:**@10.1.9.159:5672/test_host
[2020-09-10 20:00:18,595: INFO/MainProcess] mingle: searching for neighbors
[2020-09-10 20:00:19,641: INFO/MainProcess] mingle: all alone
[2020-09-10 20:00:19,658: INFO/MainProcess] celery@remote_host ready.
[2020-09-10 20:00:48,533: INFO/MainProcess] Received task: doors.routes.pushDemToDoors[d175239e-dcfe-4282-8596-7be80cf725fe]

错误日志：-

Task doors.routes.pushDemToDoors[697b9f66-d67e-41c0-972f-ca845212a1fb] succeeded in 9341.906000000075s: 
[2020-09-12 01:13:01,721: CRITICAL/MainProcess] Couldn't ack 1, reason:ConnectionResetError(10054, 'An existing connection was forcibly closed by the remote host', None, 10054, None)
Traceback (most recent call last):
  File "c:\users\g-us01.test\.virtualenvs\celery-z5n-38vt\lib\site-packages\kombu\message.py", line 131, in ack_log_error
    self.ack(multiple=multiple)
  File "c:\users\g-us01.test\.virtualenvs\celery-z5n-38vt\lib\site-packages\kombu\message.py", line 126, in ack
    self.channel.basic_ack(self.delivery_tag, multiple=multiple)
  File "c:\users\g-us01.test\.virtualenvs\celery-z5n-38vt\lib\site-packages\amqp\channel.py", line 1394, in basic_ack
    spec.Basic.Ack, argsig, (delivery_tag, multiple),
  File "c:\users\g-us01.test\.virtualenvs\celery-z5n-38vt\lib\site-packages\amqp\abstract_channel.py", line 59, in send_method
    conn.frame_writer(1, self.channel_id, sig, args, content)
  File "c:\users\g-us01.test\.virtualenvs\celery-z5n-38vt\lib\site-packages\amqp\method_framing.py", line 189, in write_frame
    write(view[:offset])
  File "c:\users\g-us01.test\.virtualenvs\celery-z5n-38vt\lib\site-packages\amqp\transport.py", line 305, in write
    self._write(s)
ConnectionResetError: [WinError 10054] An existing connection was forcibly closed by the remote host

Answer 1

我没有在 celery worker 命令中使用选项 -P gevent 发现这个问题。就我而言，连接重置问题的罪魁祸首是-P solo。

Answer 2

您的visibility timeout 设置是什么？如果您的任务运行时间超过了可见性超时设置（默认设置为 1h），您将不会得到任何东西。要将其增加到 4 小时，请执行 app.conf.broker_transport_options = {"visibility_timeout": 14400} 之类的操作。