在 elasticSearch 中创建管道的问题答案

【问题标题】：issue in creating a pipeline in elasticSearch在 elasticSearch 中创建管道的问题
【发布时间】：2021-03-24 22:42:29
【问题描述】：

我正在尝试摄取一个包含 grok、日期和删除处理器的管道，但是尽管在文档下明确提到了“消息”字段，但我收到了一个缺少字段的错误

GET _ingest/pipeline/_simulate
    {
      "pipeline" : {
        "processors" : [
          {
            "grok" : {
              "field" : "message", 
                "pattern" : "%{COMMONAPACHELOG}"
            }
          },
          {
            "date" : {
              "match_field" : "timestamp", 
              "match_formats" : ["dd/MMM/YYYY:HH:mm:ss Z"]
              
            }
          }, 
          {
          "remove" : {
            "field" :  "message" 
          }
      }
    ]
  }, 

    "docs" : [
    {
      "_source" :  {
      "message" : "52.35.38.35 -- [19/Apr/2016:12:00:04 +0200] \"GET/ HTTP/1.1\" 200 24"
      }, 
      "_index" :  "indexer" 
    }
  ]
  }

我遇到了这个错误，请帮忙

{
  "error" : {
    "root_cause" : [
      {
        "type" : "parse_exception",
        "reason" : "[patterns] required property is missing",
        "property_name" : "patterns",
        "processor_type" : "grok",
        "suppressed" : [
          {
            "type" : "parse_exception",
            "reason" : "[field] required property is missing",
            "property_name" : "field",
            "processor_type" : "date"
          }
        ]
      }
    ],
    "type" : "parse_exception",
    "reason" : "[patterns] required property is missing",
    "property_name" : "patterns",
    "processor_type" : "grok",
    "suppressed" : [
      {
        "type" : "parse_exception",
        "reason" : "[field] required property is missing",
        "property_name" : "field",
        "processor_type" : "date"
      }
    ]
  },
  "status" : 400
}

我试图在 youtube 上寻找一个视频，我发现有人使用相同的代码并且它执行得很好这是视频 https://www.youtube.com/watch?v=PEHnBa19Gxs&t=1s 现在是第 34 分钟

【问题讨论】：

标签： elasticsearch

【解决方案1】：

事实证明它适用于 youtube 的人，因为它使用的是旧版本。这将适用于较新的版本

GET _ingest/pipeline/_simulate
    {
      "pipeline" : {
        "processors" : [
          {
            "grok" : {
              
              "field" : "message", 
              "patterns" : ["%{COMMONAPACHELOG}"]
            }
          },
          {
            "date" : {
              "field" : "timestamp", 
              "formats" : ["dd/MMM/YYYY:HH:mm:ss Z"]
            }
          }, 
          {
          "remove" : {
            "field" :  "message" 
          }
      }
    ]
  }, 

    "docs" : [
    {
      "_source" :  {
      "message" : "52.35.38.35 - - [19/Apr/2016:12:00:04 +0200] \"GET/ HTTP/1.1\" 200 24"
      }, 
      "_index" :  "indexer" 
    }
  ]
  }

【讨论】：