如何使用 Elasticsearch JavaScript SDK 连接到 AWS Elasticsearch？

Question

我正在使用 AWS Elasticsearch 服务并希望通过 elasticsearch.js 进行连接，但需要端口。

看起来 AWS 只提供 REST API（例如通过 curl），在端口 80 上运行。我的集群已启动，我可以通过浏览器访问，但不能通过 elasticsearch.js。

这个例子不适合我：

var elasticsearch = require('elasticsearch');
var client = new elasticsearch.Client({
  host: 'localhost:9200', // not working: '', 80, default: 443
  log: 'trace'
});

client.ping({
  requestTimeout: 1000
}, function (error) {
  if (error) {
    console.trace('elasticsearch cluster is down!');
  } else {
    console.log('All is well');
  }
});

我找到了http-aws-es，但它也不起作用。

有什么想法吗？提前致谢！

Answer 1

对于 elasticsearch.client，您可以将 http-aws-es 用于 connectionClass 和 amazonES 与键。

var client = new elasticsearch.Client({
    hosts: config.elasticsearch.host,
    connectionClass: require('http-aws-es'),
    amazonES: {
        region: config.aws.region,
        accessKey: config.aws.key,
        secretKey: config.aws.secret
    }
});

Answer 2

NPM 包elasticsearch 已被deprecated 替换为@elastic/elasticsearch

因此，您可以考虑使用包@acuris/aws-es-connection，而不是使用http-aws-es，它应该与已弃用的elasticsearch 包一起使用，它是新elasticsearch 客户端@elastic/elasticsearch 的AWS ES 连接。它在我的项目中对我很有效。您可以在its readme file 中找到它的用法，但这里有一段示例代码：

import {
  createAWSConnection,
  awsGetCredentials,
} from '@acuris/aws-es-connection';
import { Client } from '@elastic/elasticsearch';

export const getESClient = async () => {
  const esEndpoint = process.env.AWS_ES_ENDPOINT;
  if (!esEndpoint) {
    throw new Error(
      'AWS_ES_ENDPOINT ENV not set.'
    );
  }

  const awsCredentials = await awsGetCredentials();
  const AWSConnection = createAWSConnection(awsCredentials);
  const client = new Client({
    ...AWSConnection,
    node: esEndpoint,
  });
  return client;
};

export const createNewIndex = async (index: string) => {
  try {
    const client = await getESClient();
    const exists = await client.indices.exists({ index });
    if (!exists || !exists.statusCode || exists.statusCode !== 404) {
      console.log(`Index ${index} might alrady exist.`, exists);
      return false;
    }
    const created = await client.indices.create({
      index,
      body: {
        mappings: {
          properties: {
            product_id: {
              type: 'keyword',
            },
            product_description: {
              type: 'text',
            },
          },
        },
      },
    });
    console.log(`Index created for ${index}`, created);
  } catch (error) {
    console.log(`Error creating index ${index}`, error);
    return false;
  }
  return true;
};

Answer 3

这是我一直在使用 TypeScript 的 Connection 类的实现：

import { Connection as UnsignedConnection } from '@elastic/elasticsearch';
import * as AWS from 'aws-sdk';
import RequestSigner from 'aws-sdk/lib/signers/v4';
import { ClientRequest, IncomingMessage } from 'http';

class AwsElasticsearchError extends Error {}
type RequestOptions = Parameters<UnsignedConnection['request']>[0];


class AwsSignedConnection extends UnsignedConnection {
  public request(
    params: RequestOptions,
    callback: (err: Error | null, response: IncomingMessage | null) => void,
  ): ClientRequest {
    const signedParams = this.signParams(params);
    return super.request(signedParams, callback);
  }

  private signParams(params: RequestOptions): RequestOptions {
    const region = AWS.config.region || process.env.AWS_DEFAULT_REGION;
    if (!region) throw new AwsElasticsearchError('missing region configuration');
    if (!params.method) throw new AwsElasticsearchError('missing request method');
    if (!params.path) throw new AwsElasticsearchError('missing request path');
    if (!params.headers) throw new AwsElasticsearchError('missing request headers');

    const endpoint = new AWS.Endpoint(this.url.href);
    const request = new AWS.HttpRequest(endpoint, region);

    request.method = params.method;
    request.path = params.querystring
      ? `${params.path}/?${params.querystring}`
      : params.path;
    request.body = params.body;

    Object.entries(params.headers).forEach(([header, value]) => {
      if (value === undefined) return;
      if (typeof value === 'string') request.headers[header] = value;
      else if (typeof value === 'number') request.headers[header] = `${value}`;
      else request.headers[header] = value.join('; ');
    });
    request.headers.Host = endpoint.host;

    const signer = new RequestSigner(request, 'es');
    signer.addAuthorization(AWS.config.credentials, new Date());
    return request;
  }
}

export { AwsSignedConnection, UnsignedConnection, AwsElasticsearchError };

然后你可以只在凭据可用的情况下提供它，这样你就可以使用它来指向没有凭据的本地（例如 Docker）Elasticsearch：

import awsSdk from 'aws-sdk';
import elasticsearch from '@elastic/elasticsearch';
import { AwsSignedConnection, UnsignedConnection } from '../aws-es-connector';

client = new elasticsearch.Client({
  Connection: awsSdk.config.credentials ? AwsSignedConnection : UnsignedConnection,
  node: elasticsearchEndpoint,
});

Answer 4

我花了一个小时找到适合我的方法。唯一有效的是：

var elasticsearch = require('elasticsearch');
var AWS = require('aws-sdk');
var connectionClass = require('http-aws-es');
AWS.config.update({
    credentials: new AWS.Credentials('accessKey','secret'),
    region: 'us-east-1'
  });

var client = new elasticsearch.Client({
    host: 'https://yourdomainurl.us-east-1.es.amazonaws.com',
    log: 'debug',
    connectionClass: connectionClass,
    amazonES: {
      credentials: new AWS.EnvironmentCredentials('AWS')
    }
  });

确保您为 IAM 用户添加了访问策略。那么一切都应该正常工作。 希望这会对某人有所帮助。

Answer 5

Elasticsearch 服务端点在 HTTPS（端口 443）或 HTTP（端口 80）上运行。

你可以将你的 localhost:9200 换成这个，以进行简单的替换