打开 cassandra 节点间加密会导致“无法使用任何种子进行八卦”异常答案

【问题标题】：Turning cassandra inter-node encryption on causes "Unable to gossip with any seeds" exception打开 cassandra 节点间加密会导致“无法使用任何种子进行八卦”异常
【发布时间】：2015-02-07 03:16:44
【问题描述】：

我正在尝试打开 cassandra (2.1) 节点间加密。出于测试目的，我正在尝试启动一个 2 节点集群。

我在 2 个单独的 ec2 实例上的 docker 容器内运行每个节点。如果没有节点间加密，一切都会按预期进行。

我正在使用以下脚本（取自 https://docs.jboss.org/author/display/RHQ/Cassandra+Node+To+Node+Encryption?_sscc=t）生成 ssl 密钥：

  for ((a=0; a < NUMBER_OF_NODES ; a++))
  do
     node_id=node${a}

     echo -e "Start building certificates for ${node_id}"
     echo -e "=========================================="
     rm -vf ./${node_id}.keystore
     rm -vf ./${node_id}.cer

     #1 Generate key and store
     ${java_folder}/keytool -genkey -v -keyalg RSA -keysize 1024 -alias ${node_id} -keystore ${node_id}.keystore -storepass "${node_id}store" -dname 'CN=RHQ' -keypass "${node_id}store" -validity 3650

     #2 Extract public certificate
     ${java_folder}/keytool -export -v -alias ${node_id} -file ${node_id}.cer -keystore ${node_id}.keystore -storepass "${node_id}store"

     #3 Add public certificate to global keystore
     ${java_folder}/keytool -import -v -trustcacerts -alias ${node_id} -file ${node_id}.cer -keystore global.truststore -storepass 'globalstore' -noprompt

     echo -e "========================================="
     echo -e "Done building certificates for ${node_id}\n\n"
  done

我还在每个节点的 cassandra.yml 文件中添加以下配置（node0 相应更改）：

server_encryption_options:
   internode_encryption: all
   keystore: /keystores/node0.keystore
   keystore_password: node0store
   truststore: /keystores/global.truststore
   truststore_password: globalstore

node1 配置为 node0 作为它的种子。我启动 node0，并等到它启动，我没有看到任何异常，一切都按预期工作。然后我启动node1，它会抛出以下内容（仅当调试级别设置为“trace”时）：

TRACE 08:14:16 unable to connect to 172.12.1.11/172.12.1.11
javax.net.ssl.SSLException: Unrecognized SSL message, plaintext connection?
        at sun.security.ssl.InputRecord.handleUnknownRecord(InputRecord.java:671) ~[na:1.7.0_65]
        at sun.security.ssl.InputRecord.read(InputRecord.java:504) ~[na:1.7.0_65]
        at sun.security.ssl.SSLSocketImpl.readRecord(SSLSocketImpl.java:927) ~[na:1.7.0_65]
        at sun.security.ssl.SSLSocketImpl.performInitialHandshake(SSLSocketImpl.java:1312) ~[na:1.7.0_65]
        at sun.security.ssl.SSLSocketImpl.writeRecord(SSLSocketImpl.java:702) ~[na:1.7.0_65]
        at sun.security.ssl.AppOutputStream.write(AppOutputStream.java:122) ~[na:1.7.0_65]
        at java.io.BufferedOutputStream.flushBuffer(BufferedOutputStream.java:82) ~[na:1.7.0_65]
        at java.io.BufferedOutputStream.flush(BufferedOutputStream.java:140) ~[na:1.7.0_65]
        at org.apache.cassandra.io.util.DataOutputStreamPlus.flush(DataOutputStreamPlus.java:55) ~[apache-cassandra-2.1.1.jar:2.1.1]
        at org.apache.cassandra.net.OutboundTcpConnection.connect(OutboundTcpConnection.java:347) [apache-cassandra-2.1.1.jar:2.1.1]
        at org.apache.cassandra.net.OutboundTcpConnection.run(OutboundTcpConnection.java:163) [apache-cassandra-2.1.1.jar:2.1.1]
TRACE 08:14:17 Expired 0 entries
TRACE 08:14:20 Expired 0 entries
TRACE 08:14:22 Expired 0 entries
TRACE 08:14:25 Expired 0 entries
TRACE 08:14:27 Expired 0 entries
TRACE 08:14:30 Expired 0 entries
TRACE 08:14:32 Expired 0 entries
DEBUG 08:14:34 Copy GC in 14ms.  CMS Old Gen: 9537256 -> 14901648; Eden Space: 41943040 -> 0; Survivor Space: 5242872 -> 5242880
TRACE 08:14:35 Expired 0 entries
ERROR 08:14:37 Exception encountered during startup
java.lang.RuntimeException: Unable to gossip with any seeds
        at org.apache.cassandra.gms.Gossiper.doShadowRound(Gossiper.java:1221) ~[apache-cassandra-2.1.1.jar:2.1.1]
        at org.apache.cassandra.service.StorageService.checkForEndpointCollision(StorageService.java:457) ~[apache-cassandra-2.1.1.jar:2.1.1]
        at org.apache.cassandra.service.StorageService.prepareToJoin(StorageService.java:700) ~[apache-cassandra-2.1.1.jar:2.1.1]
        at org.apache.cassandra.service.StorageService.initServer(StorageService.java:637) ~[apache-cassandra-2.1.1.jar:2.1.1]
        at org.apache.cassandra.service.StorageService.initServer(StorageService.java:529) ~[apache-cassandra-2.1.1.jar:2.1.1]
        at org.apache.cassandra.service.CassandraDaemon.setup(CassandraDaemon.java:324) [apache-cassandra-2.1.1.jar:2.1.1]
        at org.apache.cassandra.service.CassandraDaemon.activate(CassandraDaemon.java:443) [apache-cassandra-2.1.1.jar:2.1.1]
        at org.apache.cassandra.service.CassandraDaemon.main(CassandraDaemon.java:532) [apache-cassandra-2.1.1.jar:2.1.1]
java.lang.RuntimeException: Unable to gossip with any seeds
        at org.apache.cassandra.gms.Gossiper.doShadowRound(Gossiper.java:1221)
        at org.apache.cassandra.service.StorageService.checkForEndpointCollision(StorageService.java:457)
        at org.apache.cassandra.service.StorageService.prepareToJoin(StorageService.java:700)
        at org.apache.cassandra.service.StorageService.initServer(StorageService.java:637)
        at org.apache.cassandra.service.StorageService.initServer(StorageService.java:529)
        at org.apache.cassandra.service.CassandraDaemon.setup(CassandraDaemon.java:324)
        at org.apache.cassandra.service.CassandraDaemon.activate(CassandraDaemon.java:443)
        at org.apache.cassandra.service.CassandraDaemon.main(CassandraDaemon.java:532)
Exception encountered during startup: Unable to gossip with any seeds

还值得注意的是，node0 上的 7001 端口是开放的，node1 可以访问。

【问题讨论】：

此消息“无法识别的 SSL 消息，明文连接？”告诉我您正在尝试使用 SSL 连接到非 SSL 端口。您确定在 node0 上启用了加密吗？
我确实在 node0 和 node1 上看到消息“INFO 20:50:10 Starting Encrypted Messaging Service on SSL port 7001”，所以我假设 SSL 在两个节点上都打开了。
node0的IP是172.12.1.11吗？您是否在 node1 上看到相同的启动消息？

标签： ssl encryption amazon-ec2 cassandra docker

【解决方案1】：

通常情况下，问题与环境配置有关，与实际的 cassandra 设置无关。

我在 coreos 集群上的 docker 容器中运行隔离的 cassandra 实例。我忘了默认的etcd ssl端口和cassandra默认的ssl节点间通信端口都是7001。

将其中一个系统更改为使用备用端口号时，问题已解决。我认为错误消息可能更清楚（并且不需要跟踪调试级别）。更清晰的错误消息可以节省我一些时间来跟踪网络数据包以寻找答案。

【讨论】：