Mongodb：使用字段值获取文档是否是最新的并过滤结果

Question

我正在尝试将现有的 SQL 模式移植到 Mongo。
我们有文档表，有时包含多次相同的文档，具有不同的修订版但相同的参考。我只想获取文档的最新版本。

样本输入数据：

{
    "Uid" : "xxx",
    "status" : "ACCEPTED",
    "reference" : "DOC305",
    "code" : "305-D",
    "title" : "Document 305",
    "creationdate" : ISODate("2011-11-24T15:13:28.887Z"),
    "creator" : "X"
},
{
    "Uid" : "xxx",
    "status" : "COMMENTED",
    "reference" : "DOC306",
    "code" : "306-A",
    "title" : "Document 306",
    "creationdate" : ISODate("2011-11-28T07:23:18.807Z"),
    "creator" : "X"
},
{
    "Uid" : "xxx",
    "status" : "COMMENTED",
    "reference" : "DOC306",
    "code" : "306-B",
    "title" : "Document 306",
    "creationdate" : ISODate("2011-11-28T07:26:49.447Z"),
    "creator" : "X"
},
{
    "Uid" : "xxx",
    "status" : "ACCEPTED",
    "reference" : "DOC501",
    "code" : "501-A",
    "title" : "Document 501",
    "creationdate" : ISODate("2011-11-19T06:30:35.757Z"),
    "creator" : "X"
},
{
    "Uid" : "xxx",
    "status" : "ACCEPTED",
    "reference" : "DOC501",
    "code" : "501-B",
    "title" : "Document 501",
    "creationdate" : ISODate("2011-11-19T06:40:32.957Z"),
    "creator" : "X"
}

鉴于这些数据，我想要这个结果集（有时我只想要最后一个修订版，有时我想要一个属性告诉我它是否是最新的所有修订版）：

{
    "Uid" : "xxx",
    "status" : "ACCEPTED",
    "reference" : "DOC305",
    "code" : "305-D",
    "title" : "Document 305",
    "creationdate" : ISODate("2011-11-24T15:13:28.887Z"),
    "creator" : "X",
    "lastrev" : true
},
{
    "Uid" : "xxx",
    "status" : "COMMENTED",
    "reference" : "DOC306",
    "code" : "306-B",
    "title" : "Document 306",
    "creationdate" : ISODate("2011-11-28T07:26:49.447Z"),
    "creator" : "X",
    "lastrev" : true
},
{
    "Uid" : "xxx",
    "status" : "ACCEPTED",
    "reference" : "DOC501",
    "code" : "501-B",
    "title" : "Document 501",
    "creationdate" : ISODate("2011-11-19T06:40:32.957Z"),
    "creator" : "X",
    "lastrev" : true
}

我已经有一堆过滤器、排序和跳过/限制（用于数据分页），所以最终的结果集应该注意这些约束。

当前的“查找”查询（使用 .Net 驱动程序构建），可以很好地过滤，但会为我提供每个文档的所有修订：

coll.find(
    { "$and" : [
        { "$or" : [
            { "deletedid" : { "$exists" : false } },
            { "deletedid" : null }
        ] },
        { "$or" : [
            { "taskid" : { "$exists" : false } },
            { "taskid" : null }
        ] },
        { "objecttypeuid" : { "$in" : ["xxxxx"] } }
    ] },
    { "_id" : 0, "Uid" : 1, "lastrev" : 1, "title" : 1, "code" : 1, "creator" : 1, "owner" : 1, "modificator" : 1, "status" : 1, "reference": 1, "creationdate": 1 }
).sort({ "creationdate" : 1 }).skip(0).limit(10);

使用another question，我已经能够构建这个聚合，它为我提供了每个文档的最新版本，但结果中没有足够的属性：

coll.aggregate([
    { $sort: { "creationdate": 1 } },
    {
        $group: {
            "_id": "$reference",
            result: { $last: "$creationdate" },
            creationdate: { $last: "$creationdate" }
        }
    }
]);

我想将聚合与查找查询集成。

Answer 1

我找到了混合聚合和过滤的方法：

coll.aggregate(
[
    { $match: {
            "$and" : [
                { "$or" : [
                    { "deletedid" : { "$exists" : false } },
                    { "deletedid" : null }
                ] },
                { "$or" : [
                    { "taskid" : { "$exists" : false } },
                    { "taskid" : null }
                ] },
                { "objecttypeuid" : { "$in" : ["xxx"] } }
            ]
        }
    },
    { $sort: { "creationdate": 1 } },
    { $group: {
            "_id": "$reference",
            "doc": { "$last": "$$ROOT" }
        }
    },
    { $sort: { "doc.creationdate": 1 } },
    { $skip: skip },
    { $limit: limit }
],
    { allowDiskUse: true }
);

对于每个结果节点，这给了我一个带有文档数据的“doc”节点。它仍然有太多数据（缺少预测），但这是一个开始。

在.Net中翻译：

FilterDefinitionBuilder<BsonDocument> filterBuilder = Builders<BsonDocument>.Filter;
FilterDefinition<BsonDocument> filters = filterBuilder.Empty;

filters = filters & (filterBuilder.Not(filterBuilder.Exists("deletedid")) | filterBuilder.Eq("deletedid", BsonNull.Value));
filters = filters & (filterBuilder.Not(filterBuilder.Exists("taskid")) | filterBuilder.Eq("taskid", BsonNull.Value));
foreach (var f in fieldFilters) {
    filters = filters & filterBuilder.In(f.Key, f.Value);
}

var sort = Builders<BsonDocument>.Sort.Ascending(orderby);

var group = new BsonDocument {
    { "_id", "$reference" },
    { "doc", new BsonDocument("$last", "$$ROOT") }
};

var aggregate = coll.Aggregate(new AggregateOptions { AllowDiskUse = true })
    .Match(filters)
    .Sort(sort)
    .Group(group)
    .Sort(sort)
    .Skip(skip)
    .Limit(rows);

return aggregate.ToList();

不过，我很确定有更好的方法来做到这一点。

Answer 2

你的答案非常接近。 $max 比$last 更好。

关于 $last 运算符：

返回将表达式应用于按字段共享同一组的一组文档中的最后一个文档所产生的值。仅当文档按定义的顺序时才有意义。

获取每个组中的最新修订版，请参阅下面mongo shell中的代码：

db.collection.aggregate([
  {
    $group: {
      _id: '$reference',
      doc: {
        $max: {
          "creationdate" : "$creationdate",
          "code" : "$code",
          "Uid" : "$Uid",
          "status" : "$status",
          "title" : "$title",
          "creator" : "$creator"
        }
      }
    }
  },
  {
    $project: {
      _id: 0,
      Uid: "$doc.Uid",
      status: "$doc.status",
      reference: "$_id",
      code: "$doc.code",
      title: "$doc.title",
      creationdate: "$doc.creationdate",
      creator: "$doc.creator"
    }
  }
]).pretty()

如你所愿的输出：

{
    "Uid" : "xxx",
    "status" : "ACCEPTED",
    "reference" : "DOC501",
    "code" : "501-B",
    "title" : "Document 501",
    "creationdate" : ISODate("2011-11-19T06:40:32.957Z"),
    "creator" : "X"
}
{
    "Uid" : "xxx",
    "status" : "COMMENTED",
    "reference" : "DOC306",
    "code" : "306-B",
    "title" : "Document 306",
    "creationdate" : ISODate("2011-11-28T07:26:49.447Z"),
    "creator" : "X"
}
{
    "Uid" : "xxx",
    "status" : "ACCEPTED",
    "reference" : "DOC305",
    "code" : "305-D",
    "title" : "Document 305",
    "creationdate" : ISODate("2011-11-24T15:13:28.887Z"),
    "creator" : "X"
}