如何通过NodeJS调用elasticsearch apis？

Question

我的任务是对弹性搜索 api 进行 POST api 调用，

https://search-test-search-fqa4l6ubylznt7is4d5yxlmbxy.us-west-2.es.amazonaws.com/klove-ddb/recipe/_search

我以前没有任何对 AWS 服务进行 api 调用的经验。

所以，我尝试了这个 -

axios.post('https://search-test-search-fqa4l6ubylznt7is4d5yxlmbxy.us-west-2.es.amazonaws.com/klove-ddb/recipe/_search')
            .then(res => res.data)
            .then(res => console.log(res));

但我收到 {"Message":"User: anonymous is not authorized to perform: es:ESHttpPost"}

我还检查了一些 IAM 角色并将 AWSESFullAccess 策略添加到我的个人资料中。

我仍然无法解决任何问题。

请帮帮我。

Answer 1

您看到错误 User: anonymous is not authorized to perform: es:ESHttpPost 的原因是因为您在不让 ElasticSearch 知道您是谁的情况下请求数据 - 这就是它显示“匿名”的原因。

有几种身份验证方法，最简单的方法是使用elasticsearch library。使用此库，您将为该库提供一组 IAM 角色/用户的凭证（访问密钥、秘密密钥）。它将使用它来创建签名请求。已签名的请求会让 AWS 知道实际发出请求的人，因此不会以匿名方式收到请求，而是您自己。

实现此功能的另一种方法是将访问策略调整为基于 IP：

{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Effect": "Allow",
            "Principal": {
                "AWS": "*"
            },
            "Action": "es:*",
            "Condition": {
                "IpAddress": {
                    "aws:SourceIp": [
                        "AAA.BBB.CCC.DDD"
                    ]
                }
            },
            "Resource": "YOUR_ELASTICSEARCH_CLUSTER_ARN"
        }
    ]
}

此特定政策将对任何拥有您在此处提供的 ip（范围）的人开放。不过，它会让您免去签署请求的麻烦。

有助于使用 AWS ES 设置 elasticsearch-js 的库是 this one

一个工作示例如下：

const AWS = require('aws-sdk')
const elasticsearch = require('elasticsearch')
const awsHttpClient = require('http-aws-es')

let client = elasticsearch.Client({
    host: '<YOUR_ES_CLUSTER_ID>.<YOUR_ES_REGION>.es.amazonaws.com',
    connectionClass: awsHttpClient,
    amazonES: {
        region: '<YOUR_ES_REGION>',
        credentials: new AWS.Credentials('<YOUR_ACCESS_KEY>', '<YOUR_SECRET_KEY>')
    }
});

client.search({
    index: 'twitter',
    type: 'tweets',
    body: {
        query: {
            match: {
                body: 'elasticsearch'
            }
        }
    }
})
.then(res => console.log(res));

Answer 2

Elasticsearch npm 包即将被弃用，请使用 @elastic/elasticsearch 和 @acuris/aws-es-connection，这样您就不必为函数提供 IAM Credentails。

这里的代码，我用：

'use strict';

const { Client } = require('@elastic/elasticsearch');
const { createAWSConnection, awsGetCredentials } = require('@acuris/aws-es- 
connection');

module.exports.get_es_interests = async event => {
const awsCredentials = await awsGetCredentials();
const AWSConnection = createAWSConnection(awsCredentials);
const client = new Client({
...AWSConnection,
node: 'your-endpoint',
});

let bodyObj = {};
try {
    bodyObj = JSON.parse(event.body);
} catch (jsonError) {
    console.log('There was an error parsing the JSON Object', jsonError);
    return {
        statusCode: 400
    };
}

let keyword = bodyObj.keyword;

const { body } = await client.search({
index: 'index-name',
body: {
  query: {
    match: {
      name: {
        query: keyword,
        analyzer: "standard"
      }
    }
  }
}
});

var result = body.hits.hits;

return result;
};

Answer 3

现在有https://github.com/gosquared/aws-elasticsearch-js

将它们导入

const AWS = require('aws-sdk');
const ElasticSearch = require('@elastic/elasticsearch');
const { createConnector } = require('aws-elasticsearch-js');

使用可在~/.aws/config 上找到的命名配置文件配置客户端。您可以通过执行以下操作来验证这一点：cat ~/.aws/config 应该输出如下内容：

[profile work]
region=ap-southeast-2

[default]
region = ap-southeast-1

const esClient = new ElasticSearch.Client({
  nodes: [
    '<aws elastic search domain here>'
  ],
  Connection: createConnector({
    region: '<region>',
    getCreds: callback =>
      callback(
        null,
        new AWS.SharedIniFileCredentials({ profile: '<target profile>' })
      )
  })
});

然后你就可以开始使用它了：

// this query will delete all documents in an index
await esClient.delete_by_query({
  index: '<your index here>',
  body: {
    query: {
      match_all: {}
    }
  }
});

参考资料：

• https://github.com/gosquared/aws-elasticsearch-js
• https://www.elastic.co/guide/en/elasticsearch/reference/current/docs-delete-by-query.html
• https://docs.aws.amazon.com/AWSJavaScriptSDK/latest/AWS/SharedIniFileCredentials.html