Cassandra Celery python超时发生在使用django db连接执行的原始查询执行上答案

【问题标题】：Cassandra Celery python timeout happens on raw query execution using django db connection executeCassandra Celery python超时发生在使用django db连接执行的原始查询执行上
【发布时间】：2018-10-19 23:38:05
【问题描述】：

我的 celery 为 Cassandra 会话配置如下：

def cassandra_init(*args, **kwargs):
    """ Initialize a clean Cassandra connection. """
    if cql_cluster is not None:
        cql_cluster.shutdown()
    if cql_session is not None:
        cql_session.shutdown()
    connection.setup([settings.DATABASES["default"]["HOST"],], settings.DATABASES["default"]["NAME"])

# Initialize worker context (only standard tasks)
worker_process_init.connect(cassandra_init)

当我执行原始 cassandra 查询时，会发生超时，

from django.db import connection
cursor = connection.cursor()
total_ap = cursor.execute(
        "SELECT cpu_info  FROM ap_live_stats;")

它在我的 django 项目中运行良好，但在 celery 任务中却不行。

错误：

[2018-05-09 18:50:21,576: ERROR/ForkPoolWorker-5] Task apps.statistic.tasks.ap_hourly_data_migrator[77a596d4-61a2-43f4-8580-6abc6e9b5866] raised unexpected: OperationTimedOut("errors={'192.168.98.65': 'Client request timeout. See Session.execute[_async](timeout)'}, last_host=192.168.98.65",)
Traceback (most recent call last):
  File "/home/vkchlt0079/virtuals/wlc-env/lib/python3.5/site-packages/celery/app/trace.py", line 374, in trace_task
    R = retval = fun(*args, **kwargs)
  File "/home/vkchlt0079/virtuals/wlc-env/lib/python3.5/site-packages/celery/app/trace.py", line 629, in __protected_call__
    return self.run(*args, **kwargs)
  File "/home/vkchlt0079/projects/wlcd/src/web_gui/backend/django/wlcd/apps/statistic/tasks.py", line 59, in ap_hourly_data_migrator
    "SELECT cpu_info  FROM ap_live_stats;")
  File "/home/vkchlt0079/virtuals/wlc-env/lib/python3.5/site-packages/django_cassandra_engine/utils.py", line 47, in execute
    return self.cursor.execute(sql)
  File "/home/vkchlt0079/virtuals/wlc-env/lib/python3.5/site-packages/django_cassandra_engine/connection.py", line 12, in execute
    return self.connection.execute(*args, **kwargs)
  File "/home/vkchlt0079/virtuals/wlc-env/lib/python3.5/site-packages/django_cassandra_engine/connection.py", line 86, in execute
    self.session.set_keyspace(self.keyspace)
  File "cassandra/cluster.py", line 2448, in cassandra.cluster.Session.set_keyspace (cassandra/cluster.c:48048)
  File "cassandra/cluster.py", line 2030, in cassandra.cluster.Session.execute (cassandra/cluster.c:38536)
  File "cassandra/cluster.py", line 3844, in cassandra.cluster.ResponseFuture.result (cassandra/cluster.c:80834)
cassandra.OperationTimedOut: errors={'192.168.98.65': 'Client request timeout. See Session.execute[_async](timeout)'}, last_host=192.168.98.65

尝试增加超时，但不起作用并且不确定将其包含在何处。

【问题讨论】：

标签： python django cassandra celery cqlengine

【解决方案1】：

# project/tasks.py
from celery.signals import worker_init
from django.db import connection

@worker_process_init.connect
def connect_db(**kwargs):
    connection.reconnect()

这将通过 Django Cassandra 引擎启动所需的数据库连接。 Reference

【讨论】：