Spring数据聚合查询elasticsearch

Question

我正在尝试使以下弹性搜索查询与弹簧数据一起使用。目的是返回字段 "serviceName" 的唯一结果。就像 SELECT DISTINCT serviceName FROM table 与 SQL 数据库进行比较一样。

{
  "aggregations": {
    "serviceNames": {
      "terms": {
        "field": "serviceName"
      }
    }
  },
  "size":0
}

我将该字段配置为关键字，它使查询在index_name/_search api 中按照下面的响应 sn-p 完美运行：

"aggregations": {
        "serviceNames": {
            "doc_count_error_upper_bound": 0,
            "sum_other_doc_count": 0,
            "buckets": [
                {
                    "key": "service1",
                    "doc_count": 20
                },
                {
                    "key": "service2",
                    "doc_count": 8
                },
                {
                    "key": "service3",
                    "doc_count": 8
                }
            ]
        }
    }

我的问题是当我尝试使用 StringQuery 运行时，相同的查询在 Spring 数据中不起作用我收到以下错误。我猜它使用不同的 api 来运行查询。

Cannot execute jest action , response code : 400 , error : {"root_cause":[{"type":"parsing_exception","reason":"no [query] registered for [aggregations]","line":2,"col":19}],"type":"parsing_exception","reason":"no [query] registered for [aggregations]","line":2,"col":19} , message : null

我尝试使用SearchQuery 类型来实现相同的结果，没有重复，也没有对象加载，但我没有运气。下面的 sinnipet 显示了我是如何尝试这样做的。

final TermsAggregationBuilder aggregation = AggregationBuilders
                .terms("serviceName")
                .field("serviceName")
                .size(1);
        SearchQuery searchQuery = new NativeSearchQueryBuilder()
                .withIndices("index_name")
                  .withQuery(matchAllQuery())
                  .addAggregation(aggregation)
                  .withSearchType(SearchType.DFS_QUERY_THEN_FETCH)
                  .withSourceFilter(new FetchSourceFilter(new String[] {"serviceName"}, new String[] {""}))
                  .withPageable(PageRequest.of(0, 10000))
                  .build();

有人知道如何在 spring 数据上实现无对象加载和对象属性不同的聚合吗？

我尝试了很多方法来打印关于 spring 数据的查询，但我做不到，可能是因为我使用了com.github.vanroy.springdata.jest.JestElasticsearchTemplate 实现。 我得到了以下查询部分：

logger.info("query:" + searchQuery.getQuery());
logger.info("agregations:" + searchQuery.getAggregations());
logger.info("filter:" + searchQuery.getFilter());
logger.info("search type:" + searchQuery.getSearchType());

打印出来：

query:{"match_all":{"boost":1.0}}
agregations:[{"serviceName":{"terms":{"field":"serviceName","size":1,"min_doc_count":1,"shard_min_doc_count":0,"show_term_doc_count_error":false,"order":[{"_count":"desc"},{"_key":"asc"}]}}}]
filter:null
search type:DFS_QUERY_THEN_FETCH

Answer 1

我想通了，也许可以帮助某人。聚合不与查询结果一起提供，而是在它自身的结果中并且未映射到任何对象。显然出现的 Objects 结果是 elasticsearch 为运行聚合所做的查询示例（可能不确定）。 我最终创建了一个方法，该方法可以模拟 SQL SELECT DISTINCT your_column FROM your_table 上的内容，但我认为这仅适用于关键字字段，如果我没记错的话，它们有 256 个字符的限制。我解释了 cmets 中的一些行。 感谢@Val，因为我只有在调试 Jest 代码并检查生成的请求和原始响应时才能弄清楚。

public List<String> getDistinctField(String fieldName) {
    List<String> result = new ArrayList<>();

    try {
        final String distinctAggregationName = "distinct_field"; //name the aggregation

        final TermsAggregationBuilder aggregation = AggregationBuilders
                .terms(distinctAggregationName)
                .field(fieldName)
                .size(10000);//limits the number of aggregation list, mine can be huge, adjust yours

        SearchQuery searchQuery = new NativeSearchQueryBuilder()
                .withIndices("your_index")//maybe can be omitted
                .addAggregation(aggregation)
                .withSourceFilter(new FetchSourceFilter(new String[] { fieldName }, new String[] { "" }))//filter it to retrieve only the field we ar interested, probably we can take this out.
                .withPageable(PageRequest.of(0, 1))//can't be zero, and I don't want to load 10 results every time it runs, will always return one object since I found no "size":0 in query builder
                .build();
//had to use the JestResultsExtractor because com.github.vanroy.springdata.jest.JestElasticsearchTemplate don't have an implementation for ResultsExtractor, if you use Spring defaults, you can probably use it.
    final JestResultsExtractor<SearchResult> extractor = new JestResultsExtractor<SearchResult>() {
                @Override
                public SearchResult extract(SearchResult searchResult) {
                    return searchResult;
                }
            };

            final SearchResult searchResult = ((JestElasticsearchTemplate) elasticsearchOperations).query(searchQuery,
                    extractor);
            final MetricAggregation aggregations = searchResult.getAggregations();
            final TermsAggregation termsAggregation = aggregations.getTermsAggregation(distinctAggregationName);//this is where your aggregation results are, in "buckets".
            result = termsAggregation.getBuckets().parallelStream().map(TermsAggregation.Entry::getKey)
                    .collect(Collectors.toList());

        } catch (Exception e) {
            // threat your error here.
            e.printStackTrace();
        }
        return result;

    }