如何使用 elasticsearch-js 和 elasticsearch 5.0 扫描和滚动答案

【问题标题】：how do I scan and scroll with elasticsearch-js and elasticsearch 5.0如何使用 elasticsearch-js 和 elasticsearch 5.0 扫描和滚动
【发布时间】：2017-04-18 17:49:21
【问题描述】：

我之前使用elasticsearch-scrolltoend 作为插件，升级到5.0 后插件似乎不起作用。如何使用 elasticsearch 5.0 扫描和滚动大型数据集？

尝试在 elasticsearch-js docs:

json { "error": { "root_cause": [ { "type": "illegal_argument_exception", "reason": "Failed to parse request body" } ], "type": "illegal_argument_exception", "reason": "Failed to parse request body", "caused_by": { "type": "json_parse_exception", "reason": "Unrecognized token 'DnF1ZXJ5VGhlbkZldGNoBQAAAAAAAAHKFnV6a2NabEh4VDZLQmdzUzY0Y2tpd0EAAAAAAAAByxZ1emtjWmxIeFQ2S0Jnc1M2NGNraXdBAAAAAAAAAcwWdXprY1psSHhUNktCZ3NTNjRja2l3QQAAAAAAAAHOFnV6a2NabEh4VDZLQmdzUzY0Y2tpd0EAAAAAAAABzRZ1emtjWmxIeFQ2S0Jnc1M2NGNraXdB': was expecting ('true', 'false' or 'null')\n at [Source: org.elasticsearch.transport.netty4.ByteBufStreamInput@67ba4d99; line: 1, column: 457]" } }, "status": 400 }

【问题讨论】：

标签： javascript elasticsearch elasticsearch-5 elasticsearch.js

【解决方案1】：

无法识别的令牌响应错误是在13.0.0-rc1 中修复的错误——请参阅this issue for reference

关于实现，heres how I did it without the elasticsearch-scrolltoend plugin

  // methods in some ES6 class
  ...

  fetchStuff(min, max = min) {

    const body = new Bodybuilder();

    body.size(3);
    body.filter('range', 'timestamp', {
      gte: min,
      lte: max
    });

    return this.elasticsearch.search({
      index: 'my-alias',
      type: 'my-doc',
      body: body.build('v2'), // have not upgraded to newer bodybuilder package yet
      scroll: '15s'
    })
    .then((res) => this._scrollToEnd(res, []))
    .then((events) => {

      // sort by timestamp then _id
      events = _.sortBy(events, ['_.source.timestamp', '_id']);

      return events;

    });

  }

  _scrollToEnd(res, events) {
    return Promise.resolve().then(() => {

      events = events.concat(_.get(res, 'hits.hits', []));

      if (res.hits.total > events.length) {
        return this.elasticsearch.scroll({
          scrollId: res._scroll_id,
          scroll: '15s'
        })
        .then((res) => this._scrollToEnd(res, events));
      }

      return events;

    });
  }

【讨论】：