无法使用 logstash 将数据发送到 AWS 弹性搜索实例

Question

我正在尝试使用安装在我的本地计算机上的 logstash 将数据发送到 AWS 弹性搜索端点。

logstash conf 文件如下所示

input {
     file {
        path => "/path/log.txt"
      }
}
output {
   amazon_es {
        hosts => ["https://search-abclostashtrial-5jdfc43oqql7qsrhfgbvwewku.us-east-2.es.amazonaws.com"]
    action => "index"   
    region => "us-east-2"   
    index => "trial"
    ssl => true

    }
}

弹性搜索访问策略如下所示

{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Effect": "Allow",
      "Principal": {
        "AWS": "*"
      },
      "Action": "es:*",
      "Resource": "arn:aws:es:us-east-2:0415721453395:domain/abclostashtrial/*"
    }
  ]
}

我正在使用logstash-output-amazon_es 插件发送查询，例如

sudo bin/logstash -f /path/logstash/abc.conf

我得到以下错误日志。

[ERROR] 2019-04-30 20:05:52.900 [Converge PipelineAction::Create<main>] agent - Failed to execute action {:id=>:main, :action_type=>LogStash::ConvergeResult::FailedAction, :message=>"Could not execute action: PipelineAction::Create<main>, action_result: false", :backtrace=>nil} 
[INFO ] 2019-04-30 20:05:53.165 [Api Webserver] agent - Successfully started Logstash API endpoint {:port=>9600} 
[INFO ] 2019-04-30 20:05:58.037 [LogStash::Runner] runner - Logstash shut down.

我在这里错过了什么？

Answer 1

开始的一个选项是创建一个有权写入 Elasticsearch 的 AccessKey，并在输出中进行配置。示例：

amazon_es {
    hosts => ["vpc-xxxxxxxxx-es-yyyyyy4pywmwigwi47em.us-east-1.es.amazonaws.com"]
    region => "us-east-1"
    aws_access_key_id => 'AKIxxxxxxxxxxx'
    aws_secret_access_key => '11xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx'
    index => "production-logindex-%{+YYYY.MM.dd}"
    }