Zookeeper 会话不断过期...没有心跳？答案

【问题标题】：Zookeeper sessions keep expiring...no heartbeats?Zookeeper 会话不断过期...没有心跳？
【发布时间】：2016-06-22 10:27:24
【问题描述】：

我们正在使用 Kafka 高级消费者，我们能够成功消费消息，但 zookeeper 连接不断过期并重新建立。

我想知道为什么没有心跳来保持连接：

Kafka Consumer Logs
====================
 [localhost-startStop-1-SendThread(10.41.105.23:2181)] [ClientCnxn$SendThread] [line : 1096 ]  -  Client session timed out, have not heard from server in 2666ms for sessionid 0x153175bd3860159, closing socket connection and attempting reconnect
2016-03-08 18:00:06,750 INFO  [localhost-startStop-1-SendThread(10.41.105.23:2181)] [ClientCnxn$SendThread] [line : 975 ]  -  Opening socket connection to server 10.41.105.23/10.41.105.23:2181. Will not attempt to authenticate using SASL (unknown error)
2016-03-08 18:00:06,823 INFO  [localhost-startStop-1-SendThread(10.41.105.23:2181)] [ClientCnxn$SendThread] [line : 852 ]  -  Socket connection established to 10.41.105.23/10.41.105.23:2181, initiating session
2016-03-08 18:00:06,892 INFO  [localhost-startStop-1-SendThread(10.41.105.23:2181)] [ClientCnxn$SendThread] [line : 1235 ]  -  Session establishment complete on server 10.41.105.23/10.41.105.23:2181, sessionid = 0x153175bd3860159, negotiated timeout = 4000


Zookeeper Logs
==================
[2016-03-08 17:44:37,722] INFO Accepted socket connection from /10.10.113.92:51333 (org.apache.zookeeper.server.NIOServerCnxnFactory)
[2016-03-08 17:44:37,742] INFO Client attempting to renew session 0x153175bd3860159 at /10.10.113.92:51333 (org.apache.zookeeper.server.ZooKeeperServer)
[2016-03-08 17:44:37,742] INFO Established session 0x153175bd3860159 with negotiated timeout 4000 for client /10.10.113.92:51333 (org.apache.zookeeper.server.ZooKeeperServer)
[2016-03-08 17:46:56,000] INFO Expiring session 0x153175bd3860151, timeout of 4000ms exceeded (org.apache.zookeeper.server.ZooKeeperServer)
[2016-03-08 17:46:56,001] INFO Processed session termination for sessionid: 0x153175bd3860151 (org.apache.zookeeper.server.PrepRequestProcessor)
[2016-03-08 17:46:56,011] INFO Closed socket connection for client /10.10.114.183:38324 which had sessionid 0x153175bd3860151 (org.apache.zookeeper.server.NIOServerCnxn)

【问题讨论】：

我昨天刚开始在我的 Mac 上遇到类似的问题（单节点 Kafka/zookeeper 开发集群）。你在 Mac 上运行吗？
这是我的 Linux...nd 单节点集群
在 IntelliJ IDE 中以调试模式运行使我的问题更加严重。从命令行运行我们的 Scala 应用程序更加稳定
我解决了将zookeeper.connection.timeout.ms 增加到20000

标签： apache-kafka apache-zookeeper session-timeout kafka-consumer-api

【解决方案1】：

[2016-03-08 17:46:56,000] 信息到期会话 0x153175bd3860151，超过 4000 毫秒的超时 (org.apache.zookeeper.server.ZooKeeperServer)

Zookeeper 的maxSessionTimeout 是什么？如果它只有 4000 毫秒（4 秒），那么它太小了。

在Hadoop的Cloudera发行版中，ZK的maxSessionTimeout默认是40s （40000 毫秒）。

如 ZK 配置中所述 - https://zookeeper.apache.org/doc/r3.4.5/zookeeperAdmin.html 它默认为 20 ticks （一个tick 默认是2秒）。

【讨论】：

【解决方案2】：

通常 ZooKeeper 会话超时是由“软故障”引起的，这通常是垃圾收集暂停。开启 GC 日志，看看连接超时时是否发生了 long GC。另外，请阅读JVM tuning in Kafka。

【讨论】：