在spring data mongodb中如何实现分页进行聚合

Question

spring data mongodb中使用mongotemplate或mongorepository，如何实现分页进行聚合

Answer 1

这是一个旧帖子的答案，但我会提供一个答案，以防其他人在搜索此类内容时出现。

在前面的solution by Fırat KÜÇÜK 的基础上，将 results.size() 作为 PageImpl 构造函数中“total”字段的值不会使分页按您期望的分页工作的方式工作。它将总大小设置为每次的页面大小，因此，您需要找出查询将返回的实际结果总数：

public Page<UserListItemView> list(final Pageable pageable) {
    long total = getCount(<your property name>, <your property value>);

    final Aggregation agg = newAggregation(
        skip(pageable.getPageNumber() * pageable.getPageSize()),
        limit(pageable.getPageSize())
    );

    final List<UserListItemView> results = mongoTemplate
        .aggregate(agg, User.class, UserListItemView.class)
        .getMappedResults();

    return new PageImpl<>(results, pageable, total);
}

那么，现在，获得结果总数的最佳方法是另一个问题，这是我目前正在尝试解决的问题。我尝试的方法（并且有效）是几乎两次运行相同的聚合，（一次获取总计数，再次获取分页的实际结果）但仅使用 MatchOperation 后跟 GroupOperation 来获取计数：

private long getCount(String propertyName, String propertyValue) {
    MatchOperation matchOperation = match(Criteria.where(propertyName).is(propertyValue));
    GroupOperation groupOperation = group(propertyName).count().as("count");
    Aggregation aggregation = newAggregation(matchOperation, groupOperation);
    return mongoTemplate.aggregate(aggregation, Foo.class, NumberOfResults.class).getMappedResults().get(0).getCount();
}

private class NumberOfResults {
    private int count;

    public int getCount() {
        return count;
    }

    public void setCount(int count) {
        this.count = count;
    }
}

两次运行几乎相同的查询似乎效率低下，但是如果您要对结果进行分页，可分页对象必须知道结果的总数，如果您真的希望它表现得像分页。如果有人可以改进我的方法来获得结果总数，那就太棒了！

编辑：这也将提供计数，并且更简单，因为您不需要包装器对象来保存结果，因此您可以用这个替换整个前面的代码块：

private long getCount(String propertyName, String propertyValue) {
    Query countQuery = new Query(Criteria.where(propertyName).is(propertyValue));
    return mongoTemplate.count(countQuery, Foo.class);
}

Answer 2

除了ssouris solution，您还可以对结果使用Pageable 类。

public Page<UserListItemView> list(final Pageable pageable) {

    final Aggregation agg = newAggregation(
        skip(pageable.getPageNumber() * pageable.getPageSize()),
        limit(pageable.getPageSize())
    );

    final List<UserListItemView> results = mongoTemplate
        .aggregate(agg, User.class, UserListItemView.class)
        .getMappedResults();

    return new PageImpl<>(results, pageable, results.size())
}

Answer 3

你可以使用 MongoTemplate

org.spring.framework.data.mongodb.core.aggregation.Aggregation#skip
        and 
org.springframework.data.mongodb.core.aggregation.Aggregation#limit

Aggregation agg = newAggregation(
        project("tags"),
        skip(10),
        limit(10)
);

AggregationResults<TagCount> results = mongoTemplate.aggregate(agg, "tags", TagCount.class);
List<TagCount> tagCount = results.getMappedResults();

Answer 4

根据https://stackoverflow.com/a/39784851/4546949 的回答，我为 Java 编写了代码。

使用聚合组获取计数和数据数组以及其他分页信息。

    AggregationOperation group = Aggregation.group().count().as("total")
            .addToSet(pageable.getPageNumber()).as("pageNumber")
            .addToSet(pageable.getPageSize()).as("pageSize")
            .addToSet(pageable.getOffset()).as("offset")
            .push("$$ROOT").as("data");

使用聚合项目根据分页信息进行切片。

    AggregationOperation project = Aggregation.project()
            .andInclude("pageSize", "pageNumber", "total", "offset")
            .and(ArrayOperators.Slice.sliceArrayOf("data").offset((int) pageable.getOffset()).itemCount(pageable.getPageSize()))
            .as("data");

使用 mongo 模板进行聚合。

    Aggregation aggr = newAggregation(group, project);
    CustomPage page = mongoTemplate.aggregate(aggregation, Foo.class, CustomPage.class).getUniqueMappedResult();

创建自定义页面。

    public class CustomPage {
        private long pageSize;
        private long pageNumber;
        private long offset;
        private long total;
        private List<Foo> data;
    }

Answer 5

这是我的通用解决方案：

public Page<ResultObject> list(Pageable pageable) {
    // build your main stages
    List<AggregationOperation> mainStages = Arrays.asList(match(....), group(....));
    return pageAggregation(pageable, mainStages, "target-collection", ResultObject.class);
}

public <T> Page<T> pageAggregation(
        final Pageable pageable,
        final List<AggregationOperation> mainStages,
        final String collection,
        final Class<T> clazz) {
    final List<AggregationOperation> stagesWithCount = new ArrayList<>(mainStages);
    stagesWithCount.add(count().as("count"));
    final Aggregation countAgg = newAggregation(stagesWithCount);
    final Long count = Optional
            .ofNullable(mongoTemplate.aggregate(countAgg, collection, Document.class).getUniqueMappedResult())
            .map(doc -> ((Integer) doc.get("count")).longValue())
            .orElse(0L);

    final List<AggregationOperation> stagesWithPaging = new ArrayList<>(mainStages);
    stagesWithPaging.add(sort(pageable.getSort()));
    stagesWithPaging.add(skip(pageable.getOffset()));
    stagesWithPaging.add(limit(pageable.getPageSize()));
    final Aggregation resultAgg = newAggregation(stagesWithPaging);
    final List<T> result = mongoTemplate.aggregate(resultAgg, collection, clazz).getMappedResults();

    return new PageImpl<>(result, pageable, count);
}

Answer 6

要返回具有正确分页对象值的分页对象，我发现这是最好且简单的方法。

Aggregation aggregation = Aggregation.newAggregation(Aggregation.match(Criteria.where("type").is("project")),
                        Aggregation.group("id").last("id").as("id"), Aggregation.project("id"),
                        Aggregation.skip(pageable.getPageNumber() * pageable.getPageSize()),
                        Aggregation.limit(pageable.getPageSize()));


    PageableExecutionUtils.getPage(mongoTemplate.aggregate(aggregation, Draft.class, Draft.class).getMappedResults(), pageable,() -> mongoTemplate.count(Query.of(query).limit(-1).skip(-1), Draft.class));

Answer 7

另一种方法是扩展PagingAndSortingRepository<T, ID> 接口。然后，您可以像这样创建一个@Aggregation 查询方法：

@Aggregation(pipeline = {
      "{ $match: { someField: ?0 } }",
      "{ $project: { _id: 0, someField: 1} }"
})
List<StuffAggregateModel> aggregateStuff(final String somePropertyName, final Pageable pageable);

只需从您的业务逻辑服务类中调用它并构造 Pageable（如果需要，它还包含排序选项）并调用 repo 方法。我喜欢这种方法，因为它的简单性和你必须编写的代码量的绝对最小化。如果您的查询（聚合管道）足够简单，这可能是最好的解决方案。这种方法的维护编码几乎毫不费力。

Answer 8

我对 MongoDB $facet 的回答

// User(_id, first name, etc), Car (user_id, brand, etc..)
LookupOperation lookupStageCar = Aggregation.lookup(‘cars ’, ‘user_id’, ‘_id’, ‘car’);
 MatchOperation matchStage = Aggregation.match(Criteria.where(‘car.user_id ‘).exists(true));

 CountOperation countOperation = Aggregation.count().as("total");
 AddFieldsOperation addFieldsOperation = Aggregation.addFields().addFieldWithValue("page", pageable.getPageNumber()).build();
 SkipOperation skipOperation = Aggregation.skip(Long.valueOf(pageable.getPageNumber() * pageable.getPageSize()));
 LimitOperation limitOperation = Aggregation.limit(pageable.getPageSize());

// here the magic
 FacetOperation facetOperation = Aggregation.facet( countOperation, addFieldsOperation).as("metadata")
         .and(skipOperation, limitOperation).as("data");

// users with car
 List<AggrigationResults> map = mongoTemplate.aggregate(Aggregation.newAggregation( lookupStageCar, matchStage, facetOperation), "User",  AggrigationResults.class).getMappedResults();

———————————————————————————
public class AggrigationResults  {

    private List<Metadata> metadata;
    private List<User> data;

}

public class Metadata {

    private long total;
    private long page;

}

———————————————————————————
output: 
{
    "metadata" : [ 
        {
            "total" : 300,
            "page" : 3
        }
    ],
    "data" : [ 
        {
            ... original document ...
        }, 
        {
            ... another document ...
        }, 
        {
            ... etc up to 10 docs ...
        }
    ]
}

见：How to use MongoDB aggregation for pagination?