Django+Celery+postgresql_psycopg2 连接已关闭错误

Question

在强行终止（kill -9）空闲数据库连接之后。芹菜日志显示以下错误

[2016-03-08 19:28:04,765: CRITICAL/MainProcess] Task users.tasks.refresh_trend_buzz_messages[85a2430a-9a39-4979-b99d-fcaae86b3ba1] INTERNAL ERROR: InterfaceError('connection already closed',)
Traceback (most recent call last):
  File "/home/.virtualenvs/backend/local/lib/python2.7/site-packages/celery/app/trace.py", line 253, in trace_task
    I, R, state, retval = on_error(task_request, exc, uuid)
  File "/home/.virtualenvs/backend/local/lib/python2.7/site-packages/celery/app/trace.py", line 201, in on_error
    R = I.handle_error_state(task, eager=eager)
  File "/home/.virtualenvs/backend/local/lib/python2.7/site-packages/celery/app/trace.py", line 85, in handle_error_state
    }[self.state](task, store_errors=store_errors)
  File "/home/.virtualenvs/backend/local/lib/python2.7/site-packages/celery/app/trace.py", line 118, in handle_failure
    req.id, exc, einfo.traceback, request=req,
  File "/home/.virtualenvs/backend/local/lib/python2.7/site-packages/celery/backends/base.py", line 120, in mark_as_failure
    traceback=traceback, request=request)
  File "/home/.virtualenvs/backend/local/lib/python2.7/site-packages/celery/backends/base.py", line 251, in store_result
    request=request, **kwargs)
  File "/home/.virtualenvs/backend/local/lib/python2.7/site-packages/djcelery/backends/database.py", line 29, in _store_result
    traceback=traceback, children=self.current_task_children(request),
  File "/home/.virtualenvs/backend/local/lib/python2.7/site-packages/djcelery/managers.py", line 42, in _inner
    return fun(*args, **kwargs)
  File "/home/.virtualenvs/backend/local/lib/python2.7/site-packages/djcelery/managers.py", line 181, in store_result
    'meta': {'children': children}})
  File "/home/.virtualenvs/backend/local/lib/python2.7/site-packages/djcelery/managers.py", line 87, in update_or_create
    return get_queryset(self).update_or_create(**kwargs)
  File "/home/.virtualenvs/backend/local/lib/python2.7/site-packages/djcelery/managers.py", line 70, in update_or_create
    obj, created = self.get_or_create(**kwargs)
  File "/home/.virtualenvs/backend/local/lib/python2.7/site-packages/django/db/models/query.py", line 373, in get_or_create
    return self.get(**lookup), False
  File "/home/.virtualenvs/backend/local/lib/python2.7/site-packages/django/db/models/query.py", line 301, in get
    num = len(clone)
  File "/home/.virtualenvs/backend/local/lib/python2.7/site-packages/django/db/models/query.py", line 77, in __len__
    self._fetch_all()
  File "/home/.virtualenvs/backend/local/lib/python2.7/site-packages/django/db/models/query.py", line 854, in _fetch_all
    self._result_cache = list(self.iterator())
  File "/home/.virtualenvs/backend/local/lib/python2.7/site-packages/django/db/models/query.py", line 220, in iterator
    for row in compiler.results_iter():
  File "/home/.virtualenvs/backend/local/lib/python2.7/site-packages/django/db/models/sql/compiler.py", line 710, in results_iter
    for rows in self.execute_sql(MULTI):
  File "/home/.virtualenvs/backend/local/lib/python2.7/site-packages/django/db/models/sql/compiler.py", line 780, in execute_sql
    cursor = self.connection.cursor()
  File "/home/.virtualenvs/backend/local/lib/python2.7/site-packages/django/db/backends/__init__.py", line 159, in cursor
    cursor = util.CursorWrapper(self._cursor(), self)
  File "/home/.virtualenvs/backend/local/lib/python2.7/site-packages/django/db/backends/__init__.py", line 131, in _cursor
    return self.create_cursor()
  File "/home/.virtualenvs/backend/local/lib/python2.7/site-packages/django/db/utils.py", line 99, in __exit__
    six.reraise(dj_exc_type, dj_exc_value, traceback)
  File "/home/.virtualenvs/backend/local/lib/python2.7/site-packages/django/db/backends/__init__.py", line 131, in _cursor
    return self.create_cursor()
  File "/home/.virtualenvs/backend/local/lib/python2.7/site-packages/django/db/backends/postgresql_psycopg2/base.py", line 139, in create_cursor
    cursor = self.connection.cursor()
InterfaceError: connection already closed

我正在使用 Django==1.6.1、celery==3.1.10、django-celery==3.1.10 和 psycopg2==2.4.1

我该如何解决这个错误？

Answer 1

您需要在 PostgreSQL 配置中调整 max_connections。

每个 celery worker 至少需要 1 个连接，外加任何其他客户端代码的连接。

Answer 2

这是预期的行为。每个 celery worker 都会打开一个到数据库的连接并尝试重用它，以避免每次协商的开销。通过终止数据库进程，您将关闭连接，从而导致错误。

你想通过杀死数据库进程来达到什么目的？