用java提取csv中的elasticsearch查询输出答案

【问题标题】：Extract elasticsearch query output in csv with java用java提取csv中的elasticsearch查询输出
【发布时间】：2022-10-02 18:47:22
【问题描述】：

我正在使用 java 来查询 Elasticsearch；我想将我的查询数据导出到 csv 文件中。我对此感到头疼..有人可以帮我处理这段代码吗？我将感激不尽。

这是我在java中的查询代码

       try {
        RangeQueryBuilder rangeQ = QueryBuilders
                .rangeQuery(\"@timestamp\")
                .gte(\"1663632000000\")
                .lte(\"1663804799000\")
                .format(\"epoch_millis\");

        TermsAggregationBuilder termsAggregation = AggregationBuilders
                .terms(\"term_by_client_id\")
                .field(\"labels.client_id\")
                .size(100000)
                .minDocCount(1);

        termsAggregation
                .subAggregation(
                        AggregationBuilders
                                .sum(\"sum_by\")
                                .field(\"labels.row_count\")
                );
        termsAggregation
                .subAggregation(
                        AggregationBuilders
                                .terms(\"term_By_job\")
                                .field(\"labels.job_id\")
                );


        SearchRequest searchRequest = new SearchRequest();
        searchRequest.indices(\"*itm*\");


        SearchSourceBuilder searchSourceBuilder = new SearchSourceBuilder();
        searchSourceBuilder.query(rangeQ);
        searchSourceBuilder.aggregation(termsAggregation);
       // searchSourceBuilder.size(100000);
        searchRequest.source(searchSourceBuilder);

        SearchResponse searchResponse = client.search(searchRequest, RequestOptions.DEFAULT);
        System.out.println(searchResponse);

        Aggregations aggregations = searchResponse.getAggregations();
        Map<String, Aggregation> aggregationMap = aggregations.asMap();
        for (Map.Entry<String, Aggregation> each : aggregationMap.entrySet()){
            System.out.println((each.getValue()));
        }

    } catch (IOException e) {
        throw new RuntimeException(e);
    }
}

查询的输出是这样的

\"buckets\":[{\"key\":\"1741433\",\"doc_count\":1},{\"key\":\"1741435\",\"doc_count\":1} ,{\"key\":\"1741436\",\"doc_count\":1},{\"key\":\"1741440\",\"doc_count\":1},{\"key\ ":\"1741441\",\"doc_count\":1},{\"key\":\"1741442\",\"doc_count\":1},{\"key\":\"1741443\ ",\"doc_count\":1},{\"key\":\"1741444\",\"doc_count\":1},{\"key\":\"1741450\",\"doc_count\ ":1},{\"key\":\"1741451\",\"doc_count\":1}]},\"sum#sum_by\":{\"value\":1.0951264E7}},{ \"key\":\"86206\",\"doc_count\":383,\"sterms#term_By_job\":{\"doc_count_error_upper_bound\":6,\"sum_other_doc_count\":361,\"buckets\" :[{\"key\":\"1211310\",\"doc_count\":3},{\"key\":\"1211316\",\"doc_count\":3},{\"key \":\"1210943\",\"doc_count\":2},{\"key\":\"1210945\",\"doc_count\":2},{\"key\":\"1210946 \",\"doc_count\":2},{\"key\":\"1210947\",\"doc_count\":2},{\"key\":\"1210948\",\"doc_count \":2},{\"key\":\"1210949\",\"doc_count\":2},{\"key\":\"1210987\",\"doc_count\":2}, {\"key\":\"1211010\",\"doc_count\":2}]}

请在这里帮助我使用此代码，我们将不胜感激。谢谢。

您想在 csv 文件中保存哪些数据？只有这样的价值观？ 1741433,1\\n1741435,1\\n...

标签： java elasticsearch export-to-csv elasticsearch-aggregation elasticsearch-query

【解决方案1】：

这不完全是你所要求的，但我认为它可能会对你有所帮助，所以我建议它。

你可以像这样使用lightweight tool

# Backup the results of a query to a file
elasticdump \
  --input=http://production.es.com:9200/my_index \
  --output=query.json \
  --searchBody="{\"query\":{\"term\":{\"username\": \"admin\"}}}"

然后使用 Java 库或任何适合您的在线工具将 json 转换为 csv。

【讨论】：

我不想使用任何在线工具进行转换，我想在我的程序中直接实现导出代码