按术语分组并获取数组键列表及其计数

Question

在弹性搜索中，我得到了几十万个大致这种结构的文档：

{
  "script": "/index.html",
  "query": {
    "ab": "hello",
    "cd": "world",
    "ef": "123"
}

url "http://localhost/index.html?ab=hello&cd=world&ef=123" 被解析到其中。 “脚本”只包含路径和目标脚本 - 根本没有查询。 查询数组不包含相同的键列表，当然还有不同的值，这在此刻根本不重要。

我知道，我可以通过以下方式获得不同的“脚本”列表：

{
  "aggregations": {
    "my_agg": {
      "terms": {
        "field": "script.raw"
      }
    }
  }
}

这会导致多个桶，例如

"buckets": [
{
    "key": "/index.html",
    "doc_count": 123456
},
{
    "key": "/hello.html",
    "doc_count": 1456
},
...

我的问题：有没有办法额外获取所有查询keys的列表和计数，这些查询keys出现在不同的url中？

类似：

"buckets": [
{
    "key": "/index.html",
    "doc_count": 123456,
    "query_key_count": {
      "ab": 33456,
      "cd": 3456,
      "ef": 456,
      "gh": 56,
      "ij": 6
    }
},
{
    "key": "/hello.html",
    "doc_count": 1456,
    "query_key_count": {
      "zy": 156,
      "gh": 6
    }
},
...

非常感谢！！

Answer 1

要利用 Elasticsearch 的优势，您确实需要将您的文档结构化为：

{
   "script": "/index.html",
   "query": [
      {
         "query_key": "ab",
         "query_val": "hello"
      },
      {
         "query_key": "cd",
         "query_val": "world"
      },
      {
         "query_key": "ef",
         "query_val": "123"
      }
   ]
}

如果我使用nested type 设置映射：

PUT /test_index
{
   "mappings": {
      "doc": {
         "properties": {
            "query": {
               "type": "nested",
               "properties": {
                  "query_key": {
                     "type": "string",
                     "index": "not_analyzed"
                  },
                  "query_val": {
                     "type": "string",
                     "index": "not_analyzed"
                  }
               }
            },
            "script": {
               "type": "string",
               "index": "not_analyzed"
            }
         }
      }
   }
}

并添加几个文档：

POST /test_index/_bulk
{"index":{"_index":"test_index","_type":"doc","_id":1}}
{"script": "/index.html","query": [{"query_key":"ab", "query_val":"hello"},{"query_key":"cd", "query_val":"world"}, {"query_key":"ef", "query_val":"123"}]}
{"index":{"_index":"test_index","_type":"doc","_id":2}}
{"script": "/index.html","query": [{"query_key":"ab", "query_val":"foo"},{"query_key":"cd", "query_val":"bar"}, {"query_key":"gh", "query_val":"456"}]}

我可以在 nested 术语聚合中取回查询键：

POST /test_index/_search?search_type=count
{
   "aggs": {
      "resellers": {
         "nested": {
            "path": "query"
         },
         "aggs": {
            "query_keys": {
               "terms": {
                  "field": "query.query_key"
               }
            }
         }
      }
   }
}
...
{
   "took": 1,
   "timed_out": false,
   "_shards": {
      "total": 1,
      "successful": 1,
      "failed": 0
   },
   "hits": {
      "total": 2,
      "max_score": 0,
      "hits": []
   },
   "aggregations": {
      "resellers": {
         "doc_count": 6,
         "query_keys": {
            "buckets": [
               {
                  "key": "ab",
                  "doc_count": 2
               },
               {
                  "key": "cd",
                  "doc_count": 2
               },
               {
                  "key": "ef",
                  "doc_count": 1
               },
               {
                  "key": "gh",
                  "doc_count": 1
               }
            ]
         }
      }
   }
}

这是我使用的代码：

http://sense.qbox.io/gist/aecd92e5903f644e28c802860a90a86bdd7f97ee