如何在logstash中基于传导创建多个索引

Question

尝试在logstash 中为elasticsearch 创建多个索引。但是我的“如果传导”没有创建任何单个索引，如果没有传导它可以正常工作。

但是，如果我在不使用 filebeat 的情况下将输入用作文件并在 logstash 中使用，那么它可以按照我的预期工作。谁能帮我解决。

###filebeat.yml###
=============
filebeat.inputs:
- type: log
  enabled: true
  paths:
    - /home/user/vinit/pache/*.log
  fields:
    log_type: apache-log

- type: log
  enabled: true
  paths:
    - /home/user/vinit/boss/*.log
  fields:
    log_type: jboss-log
  fields_under_root: true

###pipeline-conf.conf###
==================

input {
  beats {
    port => 5044
  }
}

filter {
    grok {
           match => { "message" => "^%{IP:CLIENT_IP} (?:-|%{USER:IDEN}) (?:-|%{USER:AUTH}) \[%{HTTPDATE:CREATED_ON}\] \"(?:%{WORD:REQUEST_METHOD} (?:/|%{NOTSPACE:REQUEST})(?: HTTP/%{NUMBER:HTTP_VERSION})?|-)\" %{NUMBER:RESPONSE_CODE} (?:-|%{WORD:BYTES}) (?:-|%{WORD:EXECUTION_TIME})"}
    add_field => {
                "LOG_TYPE" => "api-log"
        }
    overwrite => [ "message" ]
    }
    grok {
           match => { "message" => "%{HTTPDATE:CREATED_ON}%{NOTSPACE}%{SPACE} (?:-|%{IP:CLIENT_IP})%{SPACE} %{NOTSPACE}(?:-|%{WORD:REQUEST_METHOD}%{SPACE}) (?:-|%{NOTSPACE:REQUEST})(?: HTTP/%{NUMBER:HTTP_VERSION})%{NOTSPACE}(?:-|%{GREEDYDATA:OTHER_INFO}) (?:-|%{NUMBER:RESPONSE_CODE}) (?:-|%{WORD:BYTES}) (?:-|%{WORD:EXECUTION_TIME})"}
    add_field => {
                "LOG_TYPE" => "web-log"
        }
    overwrite => [ "message" ]
    }
    grok {
           match => { "message" => "%{TIME:CREATED_ON}%{SPACE}\[(?<THREAD>[^\]]+)?\] %{WORD:METHOD}%{SPACE}%{JAVACLASS:CLASS} - (?<MESSAGE_LOG>[^\r\n]+)((\r?\n)(?<extra>(.|\r?\n)+))?"}
    add_field => {
                "LOG_TYPE" => "jboss-log"
        }
    overwrite => [ "message" ]
    }
}
output {
   if [fields][log_type] == "apache-log"{
     elasticsearch {
     hosts => ["localhost:9200"]
     manage_template => false
     index => "server-logs-apache"
     }
  }
   if [fields][log_type] == "jboss-log" {
     elasticsearch {
     hosts => ["localhost:9200"]
     manage_template => false
     index => "server-logs-jboss"
     }
  }
   stdout { codec => rubydebug }    
}

##Also Tried##
==============
output {
       if "apache-log" in [fields][log_type] {
         elasticsearch {
         hosts => ["localhost:9200"]
         manage_template => false
         index => "server-logs-apache"
         }
      }
       if "jboss-log" in [fields][log_type] {
         elasticsearch {
         hosts => ["localhost:9200"]
         manage_template => false
         index => "server-logs-jboss"
         }
      }
       stdout { codec => rubydebug }    
    }

我期望结果为索引：server-logs-apache, server-logs-jboss 但实际输出为空。

Answer 1

您正在添加大写的字段

add_field => { "LOG_TYPE" => "web-log" }

Elasticsearch 将不同大小写的字段分开。您应该添加字段小写 - “log_type”