范围查询中的 Java 高级 Rest 客户端内存使用情况答案

【问题标题】：Java High Level Rest Client Memory Usage In Range Queries范围查询中的 Java 高级 Rest 客户端内存使用情况
【发布时间】：2020-03-26 03:57:08
【问题描述】：

在弹性搜索上的以下函数运行范围查询，当我在多线程环境中重复使用此函数时，内存急剧增加，直到应用程序崩溃。

有没有人能解决这个问题？

/**
  RestHighLevelClient restHighLevelClient = new RestHighLevelClient( //
                RestClient.builder(new HttpHost("localhost",9200,"http")));
**/
public List<Map<String, Object>> getAllDocumentsInRange(long startTime, long endTime,
       RestHighLevelClient restHighLevelClient) {
    try {
        QueryBuilder queryBuilder = QueryBuilders//
                .rangeQuery("date")//
                .gte(startTime)//
                .lte(endTime);


        SearchSourceBuilder searchSourceBuilder = new SearchSourceBuilder();

        searchSourceBuilder.query(queryBuilder);

        searchSourceBuilder.sort(new FieldSortBuilder("date").order(SortOrder.ASC));

        searchSourceBuilder.size(10000);

        searchSourceBuilder.timeout(new TimeValue(20, TimeUnit.SECONDS));

        List<Map<String, Object>> docsMap = new ArrayList<>();

        SearchRequest searchRequest = new SearchRequest(new String[] { "mainIndex" }, searchSourceBuilder);

        Scroll scroll = new Scroll(TimeValue.timeValueSeconds(30));
        searchRequest.scroll(scroll);


        RestHighLevelClient restHighLevelClient = new RestHighLevelClient( //
                RestClient.builder(new HttpHost("localhost",9200,"http")));

        SearchResponse searchResponse = restHighLevelClient.search(searchRequest, RequestOptions.DEFAULT);

        String scrollId = searchResponse.getScrollId();

        SearchHit[] searchHits = searchResponse.getHits().getHits();

        while (searchHits != null && searchHits.length > 0) {

            for (SearchHit hit : searchHits) {
                Map<String, Object> elasticDocVersion = hit.getSourceAsMap();
                docsMap.add(elasticDocVersion);
            }

            SearchScrollRequest scrollRequest = new SearchScrollRequest(scrollId);
            scrollRequest.scroll(scroll);

            searchResponse = restHighLevelClient.scroll(scrollRequest, RequestOptions.DEFAULT);

            scrollId = searchResponse.getScrollId();

            searchHits = searchResponse.getHits().getHits();
        }

        ClearScrollRequest clearScrollRequest = new ClearScrollRequest();
        clearScrollRequest.addScrollId(scrollId);
        ClearScrollResponse clearScrollResponse = //
                restHighLevelClient.clearScroll(clearScrollRequest, RequestOptions.DEFAULT);
        boolean succeeded = clearScrollResponse.isSucceeded();
        logger.trace("search scroll clearation:{}", succeeded);
    } catch (Exception e) {
        logger.error("error in creaing QueryBuilder class: {}", e.getMessage());
    }
    return new ArrayList<>();
}

这是我运行应用程序后的内存使用情况

我尝试了不同的解决方案，比如同步上面的函数，但都没有解决问题！

【问题讨论】：

标签： elasticsearch elasticsearch-dsl elasticsearch-java-api elasticsearch-7

【解决方案1】：

不确定您使用的是什么elasticsearch客户端版本，但您可能对thiselasticsearch rest客户端内存泄漏感兴趣。

另外，我不确定传入的restHighLevelClient 是哪个，还是在方法中创建的。只有一个是有效的，但无论如何，我建议在客户端完成后调用restHighLevelClient.close() 或在try-with-resources 中使用它。

try (RestHighLevelClient restHighLevelClient = 
   new RestHighLevelClient(RestClient.builder(new HttpHost("localhost",9200,"http"))) {
...
}

【讨论】：