如何使用耐嚼的 gem 使用 elasticsearch scroll api？答案

【问题标题】：How to use elasticsearch scroll api using chewy gem?如何使用耐嚼的 gem 使用 elasticsearch scroll api？
【发布时间】：2016-07-09 13:19:41
【问题描述】：

我在我的 ROR 应用程序中使用 'chewy' gem 进行弹性搜索。但我没有找到 elasticsearch scroll api 的任何文档。当我跳转到记录的最后一页时，我遇到了错误。

[500] {"error":{"root_cause":[{"type":"query_phase_execution_exception","reason":"Result window is too
large, from + size must be less than or equal to: [10000] but was [19450]. See the scroll api for a more
efficient way to request large data sets. This limit can be set by changing the [index.max_result_window]
index level parameter."}],"type":"search_phase_execution_exception","reason":"all shards failed",
"phase":"query","grouped":true,"failed_shards":[{"shard":0,"index":"recordings","node":"tgLqH_wwRUG6NmY0PCB0nA",
"reason":{"type":"query_phase_execution_exception","reason":"Result window is too large, from + size must
 be less than or equal to: [10000] but was [19450]. See the scroll api for a more efficient way to request
 large data sets. This limit can be set by changing the [index.max_result_window] index level
 parameter."}}]},"status":500}

有没有办法在耐嚼的 gem 中实现 elasticsearch scroll api 或者他们有其他选择吗？

【问题讨论】：

看起来截至目前，2016 年 8 月 17 日，Chewy 有一个开放的错误，用于专门将 Chewy 与滚动 API 一起使用：github.com/toptal/chewy/issues/327

标签： ruby-on-rails ruby elasticsearch chewy-gem

【解决方案1】：

只要让查询大小变小，就可以批量使用滚动：

  # @example Call the `scroll` API until all the documents are returned
  #
  #     # Index 1,000 documents
  #     client.indices.delete index: 'test'
  #     1_000.times do |i| client.index index: 'test', type: 'test', id: i+1, body: {title: "Test #{i}"} end
  #     client.indices.refresh index: 'test'
  #
  #     # Open the "view" of the index by passing the `scroll` parameter
  #     # Sorting by `_doc` makes the operations faster
  #     r = client.search index: 'test', scroll: '1m', 
              body: {size: 100, sort: ['_doc']}
  #
  #     # Display the initial results
  #     puts "--- BATCH 0 -------------------------------------------------"
  #     puts r['hits']['hits'].map { |d| d['_source']['title'] }.inspect
  #
  #     # Call the `scroll` API until empty results are returned
  #     while r = client.scroll(scroll_id: r['_scroll_id'], scroll: '5m') and not r['hits']['hits'].empty? do
  #       puts "--- BATCH #{defined?($i) ? $i += 1 : $i = 1} -------------------------------------------------"
  #       puts r['hits']['hits'].map { |d| d['_source']['title'] }.inspect
  #       puts
  #     end

示例取自here，使用Elasticsearch DSL Gem

【讨论】：