Logstash / Elasticsearch：“无法安装模板”/“得到响应代码'400'”

Question

我是 ELK 堆栈的新手，我正在尝试从 logstash output.elasticsearch 安装模板，但是当我在我的 JSON 中放置“映射”键时，我遇到了这个问题：

[2020-09-12T15:19:04,321][ERROR][logstash.outputs.elasticsearch] Failed to install template. {:message=>"Got response code '400' contacting Elasticsearch at URL 'http://elasticsearch:9200/_template/maillog'", :class=>"LogStash::Outputs::ElasticSearch::HttpClient::Pool::BadResponseCodeError", :backtrace=>["/usr/share/logstash/vendor/bundle/jruby/2.5.0/gems/logstash-output-elasticsearch-10.1.0-java/lib/logstash/outputs/elasticsearch/http_client/manticore_adapter.rb:80:in `perform_request'", "/usr/share/logstash/vendor/bundle/jruby/2.5.0/gems/logstash-output-elasticsearch-10.1.0-java/lib/logstash/outputs/elasticsearch/http_client/pool.rb:291:in `perform_request_to_url'", "/usr/share/logstash/vendor/bundle/jruby/2.5.0/gems/logstash-output-elasticsearch-10.1.0-java/lib/logstash/outputs/elasticsearch/http_client/pool.rb:278:in `block in perform_request'", "/usr/share/logstash/vendor/bundle/jruby/2.5.0/gems/logstash-output-elasticsearch-10.1.0-java/lib/logstash/outputs/elasticsearch/http_client/pool.rb:373:in `with_connection'", "/usr/share/logstash/vendor/bundle/jruby/2.5.0/gems/logstash-output-elasticsearch-10.1.0-java/lib/logstash/outputs/elasticsearch/http_client/pool.rb:277:in `perform_request'", "/usr/share/logstash/vendor/bundle/jruby/2.5.0/gems/logstash-output-elasticsearch-10.1.0-java/lib/logstash/outputs/elasticsearch/http_client/pool.rb:285:in `block in Pool'", "/usr/share/logstash/vendor/bundle/jruby/2.5.0/gems/logstash-output-elasticsearch-10.1.0-java/lib/logstash/outputs/elasticsearch/http_client.rb:352:in `template_put'", "/usr/share/logstash/vendor/bundle/jruby/2.5.0/gems/logstash-output-elasticsearch-10.1.0-java/lib/logstash/outputs/elasticsearch/http_client.rb:86:in `template_install'", "/usr/share/logstash/vendor/bundle/jruby/2.5.0/gems/logstash-output-elasticsearch-10.1.0-java/lib/logstash/outputs/elasticsearch/template_manager.rb:28:in `install'", "/usr/share/logstash/vendor/bundle/jruby/2.5.0/gems/logstash-output-elasticsearch-10.1.0-java/lib/logstash/outputs/elasticsearch/template_manager.rb:16:in `install_template'", "/usr/share/logstash/vendor/bundle/jruby/2.5.0/gems/logstash-output-elasticsearch-10.1.0-java/lib/logstash/outputs/elasticsearch/common.rb:130:in `install_template'", "/usr/share/logstash/vendor/bundle/jruby/2.5.0/gems/logstash-output-elasticsearch-10.1.0-java/lib/logstash/outputs/elasticsearch/common.rb:51:in `block in setup_after_successful_connection'"]}

这是我的 JSON 模板：

{
  "index_patterns": "*-maillog-*",
  "settings": {
    "index": {
      "refresh_interval": "10s",
      "number_of_shards": 1,
      "number_of_replicas": 0
    }
  },
  "mappings": {
    "maillog": {
      "properties": {
        "ip": { "type": "ip" }
      }
    }
  }
}

这里是我的 output.elasticsearch：

output {
     elasticsearch {
        id => "test"
        index => "%{[product]}-maillog-%{+YYYY.MM.dd}"
        hosts => ["###ELASTIC_HOST###:9200"]
        document_type => "maillog"
        manage_template => true
        template_overwrite => true
        template => "${CONF_PATH}/mapping/maillog.json"
        template_name => "maillog"
      }
}

使用这个 conf elastic 无法创建我的模板，但如果我从模板中删除“映射”键，如下所示：

{
  "index_patterns": "*-maillog-*",
  "settings": {
    "index": {
      "refresh_interval": "10s",
      "number_of_shards": 1,
      "number_of_replicas": 0
    }
  }
}

没有问题了。

我的堆栈由 3 个容器组成：

elasticsearch 7.4.2

logstash 7.4.2

kibana 7.4.2

我可能遗漏了一些东西，但花了很多时间却没有解决这个问题的线索......

感谢您的帮助

Answer 1

您正在使用没有types 的elasticsearch 7.X 版。

mailog 在您的 mappings 声明之后将是您在 7.X 之前版本中的类型，但这在 7.X 版本中不再适用，您需要将您的 mappings 更改为下面的。

"mappings": {
    "properties": {
        "ip": { "type": "ip" }
    }  
}

此外，您可以在 Logstash 的 elasticsearch 输出中删除 document_type，这不再起作用。