【发布时间】：2020-06-11 16:46:05
【问题描述】：
我已经跨 2 个节点部署了 Kafka connect (2.0.0) 集群，这是 Hortonworks Data Platform 3.1.4 的一部分。它已成功部署，并且 kafka 连接也开始了。我可以使用 REST 命令使用 GET 方法列出连接器。
问题是，当我尝试在集群模式下使用 POST 方法创建连接器时，它无法创建连接器，或者如果它确实创建了连接器，则它无法创建任务。我在日志中看到以下错误
[2020-02-14 15:45:41,622] ERROR [pool-3-thread-1] [org.apache.kafka.connect.runtime.distributed.DistributedHerder] - Request to leader to reconfigure connector tasks failed
org.apache.kafka.connect.runtime.rest.errors.ConnectRestException: Failed to start RestClient: org.eclipse.jetty.io.ManagedSelector@78075cca id=0 keys=0 selected=0
at org.apache.kafka.connect.runtime.rest.RestClient.httpRequest(RestClient.java:74)
at org.apache.kafka.connect.runtime.distributed.DistributedHerder$18.run(DistributedHerder.java:1034)
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
Caused by: java.util.concurrent.RejectedExecutionException: org.eclipse.jetty.io.ManagedSelector@78075cca id=0 keys=0 selected=0
at org.eclipse.jetty.util.thread.QueuedThreadPool.execute(QueuedThreadPool.java:383)
at org.eclipse.jetty.io.SelectorManager.execute(SelectorManager.java:143)
at org.eclipse.jetty.io.ManagedSelector.doStart(ManagedSelector.java:91)
at org.eclipse.jetty.util.component.AbstractLifeCycle.start(AbstractLifeCycle.java:68)
at org.eclipse.jetty.util.component.ContainerLifeCycle.start(ContainerLifeCycle.java:131)
at org.eclipse.jetty.util.component.ContainerLifeCycle.doStart(ContainerLifeCycle.java:113)
at org.eclipse.jetty.io.SelectorManager.doStart(SelectorManager.java:273)
at org.eclipse.jetty.util.component.AbstractLifeCycle.start(AbstractLifeCycle.java:68)
at org.eclipse.jetty.util.component.ContainerLifeCycle.start(ContainerLifeCycle.java:131)
at org.eclipse.jetty.util.component.ContainerLifeCycle.doStart(ContainerLifeCycle.java:113)
at org.eclipse.jetty.client.AbstractConnectorHttpClientTransport.doStart(AbstractConnectorHttpClientTransport.java:64)
at org.eclipse.jetty.util.component.AbstractLifeCycle.start(AbstractLifeCycle.java:68)
at org.eclipse.jetty.util.component.ContainerLifeCycle.start(ContainerLifeCycle.java:131)
at org.eclipse.jetty.util.component.ContainerLifeCycle.doStart(ContainerLifeCycle.java:113)
at org.eclipse.jetty.client.HttpClient.doStart(HttpClient.java:244)
at org.eclipse.jetty.util.component.AbstractLifeCycle.start(AbstractLifeCycle.java:68)
at org.apache.kafka.connect.runtime.rest.RestClient.httpRequest(RestClient.java:71)
... 6 more
[2020-02-14 15:45:41,622] ERROR [pool-3-thread-1] [org.apache.kafka.connect.runtime.distributed.DistributedHerder] - Failed to reconfigure connector's tasks, retrying after backoff:
org.apache.kafka.connect.runtime.rest.errors.ConnectRestException: Failed to start RestClient: org.eclipse.jetty.io.ManagedSelector@78075cca id=0 keys=0 selected=0
at org.apache.kafka.connect.runtime.rest.RestClient.httpRequest(RestClient.java:74)
at org.apache.kafka.connect.runtime.distributed.DistributedHerder$18.run(DistributedHerder.java:1034)
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
Caused by: java.util.concurrent.RejectedExecutionException: org.eclipse.jetty.io.ManagedSelector@78075cca id=0 keys=0 selected=0
at org.eclipse.jetty.util.thread.QueuedThreadPool.execute(QueuedThreadPool.java:383)
at org.eclipse.jetty.io.SelectorManager.execute(SelectorManager.java:143)
at org.eclipse.jetty.io.ManagedSelector.doStart(ManagedSelector.java:91)
at org.eclipse.jetty.util.component.AbstractLifeCycle.start(AbstractLifeCycle.java:68)
at org.eclipse.jetty.util.component.ContainerLifeCycle.start(ContainerLifeCycle.java:131)
at org.eclipse.jetty.util.component.ContainerLifeCycle.doStart(ContainerLifeCycle.java:113)
at org.eclipse.jetty.io.SelectorManager.doStart(SelectorManager.java:273)
at org.eclipse.jetty.util.component.AbstractLifeCycle.start(AbstractLifeCycle.java:68)
at org.eclipse.jetty.util.component.ContainerLifeCycle.start(ContainerLifeCycle.java:131)
at org.eclipse.jetty.util.component.ContainerLifeCycle.doStart(ContainerLifeCycle.java:113)
at org.eclipse.jetty.client.AbstractConnectorHttpClientTransport.doStart(AbstractConnectorHttpClientTransport.java:64)
at org.eclipse.jetty.util.component.AbstractLifeCycle.start(AbstractLifeCycle.java:68)
at org.eclipse.jetty.util.component.ContainerLifeCycle.start(ContainerLifeCycle.java:131)
at org.eclipse.jetty.util.component.ContainerLifeCycle.doStart(ContainerLifeCycle.java:113)
at org.eclipse.jetty.client.HttpClient.doStart(HttpClient.java:244)
at org.eclipse.jetty.util.component.AbstractLifeCycle.start(AbstractLifeCycle.java:68)
at org.apache.kafka.connect.runtime.rest.RestClient.httpRequest(RestClient.java:71)
... 6 more
经过一些跟踪和错误之后，似乎在领导者连接节点上创建任务时它可以工作，如果它尝试在非领导者连接节点上创建任务，那么它会引发上述错误。不过我不太确定我的假设是否正确。
接下来，当我通过杀死其他集群节点上的进程来强制连接集群进入独立连接节点时，连接器和任务被创建工作而没有任何问题。
有人能说明一下这个问题吗，因为目前我无法在集群模式下使用 Kafka 连接。
欣赏任何想法和意见。
【问题讨论】：
标签： apache-kafka apache-kafka-connect kafka-producer-api