无法从不同的机器将数据推送到 Kafka 主题答案

【问题标题】：Not able to push data to Kafka topic from different Machine无法从不同的机器将数据推送到 Kafka 主题
【发布时间】：2019-11-20 01:36:53
【问题描述】：

我对卡夫卡很陌生。

我正在尝试将数据推送到在 172.11.22.12:9092 服务器中运行的 kafka 主题。如果我将在同一台服务器上运行生产者程序 (172.11.22.12)，我可以推送到 kafka 主题，并且我也可以在消费者控制台中看到。如果我在 Windows 机器上运行，它不会推动这个话题。生产者程序如下：

 public class SimpleKafkaProducer {
        public static void main(String args[]) {
            Properties props = new Properties();
             props.put("bootstrap.servers", "172.11.22.12:9092");
             props.put("acks", "all");
             props.put("retries", 0);
             props.put("batch.size", 16384);
             props.put("linger.ms", 1);
             props.put("buffer.memory", 33554432);
             props.put("key.serializer", "org.apache.kafka.common.serialization.StringSerializer");
             props.put("value.serializer", "org.apache.kafka.common.serialization.StringSerializer");

             org.apache.kafka.clients.producer.Producer<String, String> producer = new org.apache.kafka.clients.producer.KafkaProducer<>(props);
             for(int i = 1; i < 100; i++)
                producer.send(new ProducerRecord<String, String>("test", Integer.toString(i), Integer.toString(i)));

             producer.close();

        }

    }

日志如下

14:46:47.662 [main] DEBUG o.a.k.c.producer.internals.Metadata - Updated cluster metadata version 1 to Cluster(nodes = [Node(172.11.22.12, 9092)], partitions = [])
14:46:47.771 [kafka-producer-network-thread | producer-1] DEBUG o.a.k.c.producer.internals.Sender - Starting Kafka producer I/O thread.
14:46:47.771 [main] DEBUG o.a.k.clients.producer.KafkaProducer - Kafka producer started
14:46:47.773 [kafka-producer-network-thread | producer-1] DEBUG o.apache.kafka.clients.NetworkClient - Trying to send metadata request to node -1
14:46:47.773 [kafka-producer-network-thread | producer-1] DEBUG o.apache.kafka.clients.NetworkClient - Init connection to node -1 for sending metadata request in the next iteration
14:46:47.773 [kafka-producer-network-thread | producer-1] DEBUG o.apache.kafka.clients.NetworkClient - Initiating connection to node -1 at 172.11.22.12:9092.
14:46:47.776 [kafka-producer-network-thread | producer-1] DEBUG o.apache.kafka.clients.NetworkClient - Trying to send metadata request to node -1
14:46:47.876 [kafka-producer-network-thread | producer-1] DEBUG o.apache.kafka.clients.NetworkClient - Trying to send metadata request to node -1
14:46:47.977 [kafka-producer-network-thread | producer-1] DEBUG o.apache.kafka.clients.NetworkClient - Trying to send metadata request to node -1
14:46:48.077 [kafka-producer-network-thread | producer-1] DEBUG o.apache.kafka.clients.NetworkClient - Trying to send metadata request to node -1
14:46:48.177 [kafka-producer-network-thread | producer-1] DEBUG o.apache.kafka.clients.NetworkClient - Trying to send metadata request to node -1
14:46:48.277 [kafka-producer-network-thread | producer-1] DEBUG o.apache.kafka.clients.NetworkClient - Trying to send metadata request to node -1
14:46:48.378 [kafka-producer-network-thread | producer-1] DEBUG o.apache.kafka.clients.NetworkClient - Trying to send metadata request to node -1
14:46:48.478 [kafka-producer-network-thread | producer-1] DEBUG o.apache.kafka.clients.NetworkClient - Trying to send metadata request to node -1
14:46:48.578 [kafka-producer-network-thread | producer-1] DEBUG o.apache.kafka.clients.NetworkClient - Trying to send metadata request to node -1
14:46:48.678 [kafka-producer-network-thread | producer-1] DEBUG o.apache.kafka.clients.NetworkClient - Trying to send metadata request to node -1
14:46:48.778 [kafka-producer-network-thread | producer-1] DEBUG o.apache.kafka.clients.NetworkClient - Trying to send metadata request to node -1
14:46:48.780 [kafka-producer-network-thread | producer-1] WARN  o.a.kafka.common.network.Selector - Error in I/O with /172.11.22.12
14:46:48.781 [kafka-producer-network-thread | producer-1] DEBUG o.apache.kafka.clients.NetworkClient - Node -1 disconnected.
14:46:48.879 [kafka-producer-network-thread | producer-1] DEBUG o.apache.kafka.clients.NetworkClient - Trying to send metadata request to node -1
14:46:48.879 [kafka-producer-network-thread | producer-1] DEBUG o.apache.kafka.clients.NetworkClient - Init connection to node -1 for sending metadata request in the next iteration
14:46:48.879 [kafka-producer-network-thread | producer-1] DEBUG o.apache.kafka.clients.NetworkClient - Initiating connection to node -1 at 172.11.22.12:9092.
14:46:48.879 [kafka-producer-network-thread | producer-1] DEBUG o.apache.kafka.clients.NetworkClient - Trying to send metadata request to node -1
14:46:48.979 [kafka-producer-network-thread | producer-1] DEBUG o.apache.kafka.clients.NetworkClient - Trying to send metadata request to node -1
14:46:49.079 [kafka-producer-network-thread | producer-1] DEBUG o.apache.kafka.clients.NetworkClient - Trying to send metadata request to node -1
14:46:49.179 [kafka-producer-network-thread | producer-1] DEBUG o.apache.kafka.clients.NetworkClient - Trying to send metadata request to node -1
14:46:49.280 [kafka-producer-network-thread | producer-1] DEBUG o.apache.kafka.clients.NetworkClient - Trying to send metadata request to node -1
14:46:49.380 [kafka-producer-network-thread | producer-1] DEBUG o.apache.kafka.clients.NetworkClient - Trying to send metadata request to node -1
14:46:49.480 [kafka-producer-network-thread | producer-1] DEBUG o.apache.kafka.clients.NetworkClient - Trying to send metadata request to node -1
14:46:49.580 [kafka-producer-network-thread | producer-1] DEBUG o.apache.kafka.clients.NetworkClient - Trying to send metadata request to node -1
14:46:49.680 [kafka-producer-network-thread | producer-1] DEBUG o.apache.kafka.clients.NetworkClient - Trying to send metadata request to node -1
14:46:49.781 [kafka-producer-network-thread | producer-1] DEBUG o.apache.kafka.clients.NetworkClient - Trying to send metadata request to node -1
14:46:49.881 [kafka-producer-network-thread | producer-1] DEBUG o.apache.kafka.clients.NetworkClient - Trying to send metadata request to node -1
14:46:49.884 [kafka-producer-network-thread | producer-1] WARN  o.a.kafka.common.network.Selector - Error in I/O with /172.11.22.12

我们将不胜感激。

【问题讨论】：

我使用的是apache kafka 0.10.0.0版本
通过改变server.properties的配置advertised.listeners解决。
嗨，类似的情况和同样的问题。请指导，你的卡夫卡经纪人只在“机器 172.11.22.12”，对吗？而您只将“advertised.listeners=PLAINTEXT://137.226.58.50:9092”这一行添加到“172.11.22.12 上的 server.properties”中？提前致谢！

标签： java apache-kafka kafka-producer-api

【解决方案1】：

您需要在关闭之前刷新生产者，如下所示：

public static void main(String args[]) {
            Properties props = new Properties();
             props.put("bootstrap.servers", "172.11.22.12:9092");
             props.put("acks", "all");
             props.put("retries", 0);
             props.put("batch.size", 16384);
             props.put("linger.ms", 1);
             props.put("buffer.memory", 33554432);
             props.put("key.serializer", "org.apache.kafka.common.serialization.StringSerializer");
             props.put("value.serializer", "org.apache.kafka.common.serialization.StringSerializer");

             org.apache.kafka.clients.producer.Producer<String, String> producer = new org.apache.kafka.clients.producer.KafkaProducer<>(props);
             for(int i = 1; i < 100; i++)
                producer.send(new ProducerRecord<String, String>("test", Integer.toString(i), Integer.toString(i)));

             producer.flush();
             producer.close();

        }

【讨论】：