Kafka 通过 Hyperledger Fabric Orderer 连接提供无效的接收大小答案

【问题标题】：Kafka gives Invalid receive size with Hyperledger Fabric Orderer connectionKafka 通过 Hyperledger Fabric Orderer 连接提供无效的接收大小
【发布时间】：2019-09-15 00:31:21
【问题描述】：

我在 EKS 上为 Hyperledger Fabric 设置了一个新集群。集群有4个kafka节点，3个zookeeper节点，4个peer，3个orderer，1个CA。所有容器都是单独出现的，kafka/zookeeper 后端也很稳定。我可以通过 SSH 连接到任何 kafka/zookeeper 并检查与任何其他节点的连接、创建主题、发布消息等。所有订购者都可以通过 Telnet 访问 kafka。

当我尝试创建频道时，我从订购者那里收到以下错误：

2019-04-25 13:34:17.660 UTC [orderer.common.broadcast] ProcessMessage -> WARN 025 [channel: channel1] Rejecting broadcast of message from 192.168.94.15:53598 with SERVICE_UNAVAILABLE: rejected by Consenter: backing Kafka cluster has not completed booting; try again later
2019-04-25 13:34:17.660 UTC [comm.grpc.server] 1 -> INFO 026 streaming call completed grpc.service=orderer.AtomicBroadcast grpc.method=Broadcast grpc.peer_address=192.168.94.15:53598 grpc.code=OK grpc.call_duration=14.805833ms
2019-04-25 13:34:17.661 UTC [common.deliver] Handle -> WARN 027 Error reading from 192.168.94.15:53596: rpc error: code = Canceled desc = context canceled
2019-04-25 13:34:17.661 UTC [comm.grpc.server] 1 -> INFO 028 streaming call completed grpc.service=orderer.AtomicBroadcast grpc.method=Deliver grpc.peer_address=192.168.94.15:53596 error="rpc error: code = Canceled desc = context canceled" grpc.code=Canceled grpc.call_duration=24.987468ms

并且kafka leader报如下错误：

[2019-04-25 14:07:09,453] WARN [SocketServer brokerId=2] Unexpected error from /192.168.89.200; closing connection (org.apache.kafka.common.network.Selector)
org.apache.kafka.common.network.InvalidReceiveException: Invalid receive (size = 369295617 larger than 104857600)
        at org.apache.kafka.common.network.NetworkReceive.readFromReadableChannel(NetworkReceive.java:132)
        at org.apache.kafka.common.network.NetworkReceive.readFrom(NetworkReceive.java:93)
        at org.apache.kafka.common.network.KafkaChannel.receive(KafkaChannel.java:231)
        at org.apache.kafka.common.network.KafkaChannel.read(KafkaChannel.java:192)
        at org.apache.kafka.common.network.Selector.attemptRead(Selector.java:528)
        at org.apache.kafka.common.network.Selector.pollSelectionKeys(Selector.java:469)
        at org.apache.kafka.common.network.Selector.poll(Selector.java:398)
        at kafka.network.Processor.poll(SocketServer.scala:535)
        at kafka.network.Processor.run(SocketServer.scala:452)
        at java.lang.Thread.run(Thread.java:748)
[2019-04-25 14:13:53,917] INFO [GroupMetadataManager brokerId=2] Removed 0 expired offsets in 0 milliseconds. (kafka.coordinator.group.GroupMetadataManager)

【问题讨论】：

标签： apache-kafka hyperledger-fabric

【解决方案1】：

该错误表明您收到的消息大于允许的最大大小，默认为 ~100MB。尝试在server.properties 文件中增加以下属性，使其适合更大的接收（在这种情况下至少为 369295617 字节）：

# Set to 500MB
socket.request.max.bytes=500000000

然后重新启动您的 Kafka 集群。

如果这对您不起作用，那么我猜您正在尝试连接到非 SSL 侦听器。因此，您必须验证代理的 SSL 侦听器端口是 9092（或相应端口，以防您不使用默认端口）。以下应该可以解决问题：

listeners=SSL://:9092
advertised.listeners=SSL://:9092
inter.broker.listener.name=SSL

【讨论】：