Elasticsearch 过滤器仅显示具有列表值的文档

Question

通过以下脚本，我将 elasticsearch 设置为索引文章。每篇文章都可以有一个作者列表：

#!/usr/bin/env bash 
HOST='http://localhost:9200/articles'

curl -XPUT ${HOST} 
curl -XPUT "${HOST}/article/_mapping" -d '{
    "article" : {
        "properties" : {
            "authors" : {"type":"string", "index_name":"author", "index": "not_analyzed"}
        }
    } 
}' 
curl -XPOST  "${HOST}/article/1"  -d '{
    "authors" : ["Albert","Wolfgang","Richard","Murray"],
    "message" : "Blabla" }' 
curl -XPOST  "${HOST}/article/2"  -d '{
    "authors" : ["Albert","Richard"],
    "message" : "Blublu" }' 
curl -XPOST  "${HOST}/article/3"  -d '{
    "authors" : ["Albert"],
    "message" : "Bleble" }'

我想要做的是过滤掉所有作者不在给定列表中的文章。我尝试了以下查询：

curl -XGET  "${HOST}/_search?pretty=true" -d '{
    "query": {
        "constant_score": {
            "filter": {
                "terms": { 
                    "authors": ["Albert","Richard","Erwin"],
                    "execution": "or" 
                }            
            }
        }
    } 
}'

但是，这会将所有三篇文章都返回为命中。但我确实想过滤掉文章 1，因为它有一些作者["Wolfgang", "Murray"]，它们不在给定作者["Albert", "Richard","Erwin"] 的列表中。这可以通过 elasticsearch 以某种方式实现吗？

Answer 1

棘手的一个。如果我理解得很好，您需要找到 author 字段也包含仅这三个值中的一个（或多个）的文档

根据 elasticsearch 权威指南here，检查一个字段是否只包含一些值是相当昂贵的。

有一个受equality paragraph 启发的丑陋解决方案，它将使用布尔过滤器和添加作者计数字段的组合。

您必须检查是否：

• 作者包含 Albert/Richard/Erwin 之一，并且 author_count 等于 1
• author 包含 2 个作者（Albert & Richard、Erwin & Albert 等）的可能组合之一，并且 author_count 为 2
• author 包含 3 个作者，并且 author_count 为 3

这对我来说不可行。

Answer 2

正如 Tom83 和链接的 elasticsearch 权威指南所建议的，这可以通过在索引时添加 authors_counts 属性来完成。像这样：

curl -XPOST  "${HOST}/article/1"  -d '{
  "authors": [
    "Albert",
    "Wolfgang",
    "Richard",
    "Murray"
  ],
  "authors_counts": 4,
  "message": "Blabla"
}'

curl -XPOST  "${HOST}/article/2"  -d '{
  "authors": [
    "Albert",
    "Richard"
  ],
  "authors_counts": 2,
  "message": "Blublu"
}'

curl -XPOST  "${HOST}/article/3"  -d '{
  "authors": [
    "Albert"
  ],
  "authors_counts": 1,
  "message": "Bleble"
}'

然后，您可以按照指南中的建议创建查询。它非常冗长，所以我决定生成它。

(ns query-gen 
  (:require [clojure.data.json :as json]
            [clojure.math.combinatorics :refer [subsets]]))

(defn gen-filter [items]
  (let [terms (map (fn [term] { "term" { "authors" term } }) items)
        terms_count { "term" { "authors_counts" (count items) }}] 
      { "bool" { "must" (cons terms_count terms)}}))

(defn gen-query [names]
  (let [subsets (rest (subsets names))
        filters (map gen-filter subsets)]
    {"query" { "filtered" { "filter" { "or" filters }}}}))

(defn -main [& args]
  (let [ query (gen-query ["Albert" "Richard" "Erwin"])
         json (json/write-str query)]
      (println json)))

这会产生如下所示的查询：

{
  "query": {
    "filtered": {
      "filter": {
        "or": [
          {
            "bool": {
              "must": [
                {
                  "term": {
                    "authors_counts": 1
                  }
                },
                {
                  "term": {
                    "authors": "Albert"
                  }
                }
              ]
            }
          },
          {
            "bool": {
              "must": [
                {
                  "term": {
                    "authors_counts": 1
                  }
                },
                {
                  "term": {
                    "authors": "Richard"
                  }
                }
              ]
            }
          },
          {
            "bool": {
              "must": [
                {
                  "term": {
                    "authors_counts": 1
                  }
                },
                {
                  "term": {
                    "authors": "Erwin"
                  }
                }
              ]
            }
          },
          {
            "bool": {
              "must": [
                {
                  "term": {
                    "authors_counts": 2
                  }
                },
                {
                  "term": {
                    "authors": "Albert"
                  }
                },
                {
                  "term": {
                    "authors": "Richard"
                  }
                }
              ]
            }
          },
          {
            "bool": {
              "must": [
                {
                  "term": {
                    "authors_counts": 2
                  }
                },
                {
                  "term": {
                    "authors": "Albert"
                  }
                },
                {
                  "term": {
                    "authors": "Erwin"
                  }
                }
              ]
            }
          },
          {
            "bool": {
              "must": [
                {
                  "term": {
                    "authors_counts": 2
                  }
                },
                {
                  "term": {
                    "authors": "Richard"
                  }
                },
                {
                  "term": {
                    "authors": "Erwin"
                  }
                }
              ]
            }
          },
          {
            "bool": {
              "must": [
                {
                  "term": {
                    "authors_counts": 3
                  }
                },
                {
                  "term": {
                    "authors": "Albert"
                  }
                },
                {
                  "term": {
                    "authors": "Richard"
                  }
                },
                {
                  "term": {
                    "authors": "Erwin"
                  }
                }
              ]
            }
          }
        ]
      }
    }
  }
}

如果像这样使用：

curl -XGET  "${HOST}/_search?pretty=true" -d '{
  "query": {
    "filtered": {
      "filter": {
        "or": [
          {
            "bool": {
              "must": [
                {
                  "term": {
                    "authors_counts": 1
                  }
                },
                {
                  "term": {
                    "authors": "Albert"
                  }
                }
              ]
            }
          },
          {
            "bool": {
              "must": [
                {
                  "term": {
                    "authors_counts": 1
                  }
                },
                {
                  "term": {
                    "authors": "Richard"
                  }
                }
              ]
            }
          },
          {
            "bool": {
              "must": [
                {
                  "term": {
                    "authors_counts": 1
                  }
                },
                {
                  "term": {
                    "authors": "Erwin"
                  }
                }
              ]
            }
          },
          {
            "bool": {
              "must": [
                {
                  "term": {
                    "authors_counts": 2
                  }
                },
                {
                  "term": {
                    "authors": "Albert"
                  }
                },
                {
                  "term": {
                    "authors": "Richard"
                  }
                }
              ]
            }
          },
          {
            "bool": {
              "must": [
                {
                  "term": {
                    "authors_counts": 2
                  }
                },
                {
                  "term": {
                    "authors": "Albert"
                  }
                },
                {
                  "term": {
                    "authors": "Erwin"
                  }
                }
              ]
            }
          },
          {
            "bool": {
              "must": [
                {
                  "term": {
                    "authors_counts": 2
                  }
                },
                {
                  "term": {
                    "authors": "Richard"
                  }
                },
                {
                  "term": {
                    "authors": "Erwin"
                  }
                }
              ]
            }
          },
          {
            "bool": {
              "must": [
                {
                  "term": {
                    "authors_counts": 3
                  }
                },
                {
                  "term": {
                    "authors": "Albert"
                  }
                },
                {
                  "term": {
                    "authors": "Richard"
                  }
                },
                {
                  "term": {
                    "authors": "Erwin"
                  }
                }
              ]
            }
          }
        ]
      }
    }
  }
}'

它返回我认为你期望的结果：

{
  "took": 5,
  "timed_out": false,
  "_shards": {
    "total": 5,
    "successful": 5,
    "failed": 0
  },
  "hits": {
    "total": 2,
    "max_score": 1.0,
    "hits": [
      {
        "_index": "articles",
        "_type": "article",
        "_id": "3",
        "_score": 1.0,
        "_source": {
          "authors": [
            "Albert"
          ],
          "authors_counts": 1,
          "message": "Bleble"
        }
      },
      {
        "_index": "articles",
        "_type": "article",
        "_id": "2",
        "_score": 1.0,
        "_source": {
          "authors": [
            "Albert",
            "Richard"
          ],
          "authors_counts": 2,
          "message": "Blublu"
        }
      }
    ]
  }
}

我不确定这是否是个好主意，但很有趣。希望对您有所帮助！