datastax python驱动程序连接超时与多处理答案

【问题标题】：datastax python driver connection time-out with multiprocessingdatastax python驱动程序连接超时与多处理
【发布时间】：2014-06-14 04:26:06
【问题描述】：

下面的例子：

from cassandra.cluster import Cluster
from cassandra.decoder import *
import multiprocessing

def async_query_process(lock, i, process_num):

    if i < process_num:
        lock.acquire()
        cluster = Cluster(['neptune'])
        session = cluster.connect('mercure')
        session.row_factory = tuple_factory 
        lock.release()
        process = multiprocessing.Process(name='process' + str(i+1),
                                        target=async_query_process,
                                        args=(lock, i+1, 2))
    process.start()

lock = multiprocessing.Lock()
process = multiprocessing.Process(name='process' + str(0), target=async_query_process,# cluster_ip_list = self.cluster_ip_list, keyspace=self.keyspace,
                                  args=(lock, 0, 2))
process.start()

给出以下错误：

Traceback (most recent call last):
  File "/usr/lib/python2.7/multiprocessing/process.py", line 258, in _bootstrap
    self.run()
  File "/usr/lib/python2.7/multiprocessing/process.py", line 114, in run
    self._target(*self._args, **self._kwargs)
  File "test_api.py", line 14, in async_query_process
    session = cluster.connect('mercure')
  File "/home/nkou/workProd/local/lib/python2.7/site-packages/cassandra/cluster.py", line 483, in connect
    self.control_connection.connect()
  File "/home/nkou/workProd/local/lib/python2.7/site-packages/cassandra/cluster.py", line 1369, in connect
    self._set_new_connection(self._reconnect_internal())
  File "/home/nkou/workProd/local/lib/python2.7/site-packages/cassandra/cluster.py", line 1405, in _reconnect_internal
    raise NoHostAvailable("Unable to connect to any servers", errors)
NoHostAvailable: ('Unable to connect to any servers', {'neptune': OperationTimedOut('errors=Timed out creating new connection, last_host=None',)})

datastax python驱动文档中提到集群和会话实例应该在fork之后创建，这就是为什么我必须使用那种嵌套的子进程结构。

我有点卡在这里，也许我会切换回以前工作的 cassandra-python 驱动程序。

谢谢！

尼可

【问题讨论】：

您是否尝试在端口 9042 上远程登录到海王星？您是否尝试过将 neptune 替换为服务器的 IP 以查看是否可行？
嗨，亚历克斯！抱歉我的回答延迟：是的，我试过了，它没有任何改变。

标签： cassandra multiprocessing

【解决方案1】：

这将起作用：（在启动新进程之前关闭会话）

def async_query_process(lock, i, process_num):

if i < process_num:
    lock.acquire()
    cluster = Cluster(['neptune'])
    session = cluster.connect('mercure')
    session.row_factory = tuple_factory
    session.cluster.shutdown()    # close session here
    session.shutdown() 
    lock.release()
    process = multiprocessing.Process(name='process' + str(i+1),
                                    target=async_query_process,
                                    args=(lock, i+1, 2))
process.start()

【讨论】：

当你关机时，你以后不需要在某个地方“重新启动”吗？不确定关机的作用以及以后是否需要进一步的操作。
shutdown() 关闭与此集群关联的所有会话和连接。为确保正确关闭所有连接，您应该始终在完成集群实例后调用 shutdown()。一旦关闭，集群不应用于任何目的。
关闭一个会话并打开一个新会话会非常慢，恕我直言。