Winlogbeat à kafka à logstash à elasticsearch
1. Winlogbeat
Winlogbeat 配置：
| output.kafka: | |
| # initial brokers for reading cluster metadata |
|
| # Place your HELK IP(s) here (keep the port). |
|
| # If you only have one Kafka instance (default for HELK) then remove the 2nd IP that has port 9093 |
|
| hosts: ["<HELK-IP>:9092","<HELK-IP>:9093"] |
|
| topic: "winlogbeat" |
|
| ############################# HELK Optimizing Latency ###################### |
|
| max_retries: 2 |
|
| max_message_bytes: 1000000 |
Winlogbeat 会将采集的数据 写入kafka broker的topic winlogbeat。
2. Kafka
在docker-compose文件中，会设定需要创建的topic，
在kafka-create-topics.sh，会根据变量$KAFKA_CREATE_TOPICS创建相应的topic
也就是说，是kafka先创建topic，然后winlogbeat将消息写入该topic。
3. Logstash
在pipeline配置文件0002-kafka-input.conf中，
Logstash从kafka broker中读取topics
["winlogbeat","winevent","SYSMON_JOIN","filebeat"] 中的消息