时间序列和聚合框架（mongodb）

Question

我正在尝试同步我在应用程序中运行的两个函数。 第一个实时检查我在每个时间块（例如每 10 秒）中保存到 MongoDB 的文档数：

var getVolume = function(timeBlock, cb) {
    var triggerTime = Date.now();
    var blockPeriod = triggerTime - timeBlock;

    Document.find({
        time: { $gt: blockPeriod }
    }).count(function(err, count) {
        log('getting volume since ', new Date(blockPeriod), 'result is', count)
        cb(triggerTime, count);
    });
};

然后我有了第二个函数，每当我想为我的图表（前端）获取数据时，我都会使用它：

var getHistory = function(timeBlock, end, cb) {

    Document.aggregate(
    {
        $match: {
            time: {
                $gte: new Date(end - 10 * timeBlock),
                $lt: new Date(end)
            }
        }
    },

    // count number of documents based on time block
    // timeBlock is divided by 1000 as we use it as seconds here
    // and the timeBlock parameter is in miliseconds
    {
        $group: {
            _id: {
                year: { $year: "$time" },
                month: { $month: "$time" },
                day: { $dayOfMonth: "$time" },
                hour: { $hour: "$time" },
                minute: { $minute: "$time" },
                second: { $subtract: [
                    { $second: "$time" },
                    { $mod: [
                        { $second: "$time" },
                        timeBlock / 1000
                    ]}
                ]}
            },
            count: { $sum: 1 }
        }
    },

    // changing the name _id to timeParts
    {
        $project: {
            timeParts: "$_id",
            count: 1,
            _id: 0
        }
    },

    // sorting by date, from earliest to latest
    {
        $sort: {
            "time": 1
        }
    }, function(err, result) {
        if (err) {
            cb(err)
        } else {
            log("start", new Date(end - 10 * timeBlock))
            log("end", new Date(end))
            log("timeBlock", timeBlock)
            log(">****", result)
            cb(result)
        }
    })
}

问题是我无法在图表和后端代码（getVolume 函数）上获得相同的值

我意识到来自getHistory 的日志不是我所期望的（日志如下）：

start Fri Jul 18 2014 11:56:56 GMT+0100 (BST)
end Fri Jul 18 2014 11:58:36 GMT+0100 (BST)
timeBlock 10000
>**** [ { count: 4,
    timeParts: { year: 2014, month: 7, day: 18, hour: 10, minute: 58, second: 30 } },
  { count: 6,
    timeParts: { year: 2014, month: 7, day: 18, hour: 10, minute: 58, second: 20 } },
  { count: 3,
    timeParts: { year: 2014, month: 7, day: 18, hour: 10, minute: 58, second: 10 } },
  { count: 3,
    timeParts: { year: 2014, month: 7, day: 18, hour: 10, minute: 58, second: 0 } },
  { count: 2,
    timeParts: { year: 2014, month: 7, day: 18, hour: 10, minute: 57, second: 50 } } ]

所以我希望getHistory 应该从start Fri Jul 18 2014 11:56:56 GMT+0100 (BST) 开始每 10 秒在 mongo 中查找数据，所以它看起来大致如下：

11:56:56 count: 3
11:57:06 count: 0
11:57:16 count: 14
... etc.

待办事项： 1.我知道我应该在我的聚合函数中覆盖计数为0的情况，我猜这个时间被跳过了`

Answer 1

您的错误是您如何为$group 运算符计算_id，特别是它的second 部分：

second: { $subtract: [
    { $second: "$time" },
    { $mod: [
        { $second: "$time" },
        timeBlock / 1000
    ]}
]}

因此，不是将所有数据拆分为从 new Date(end - 10 * timeBlock) 开始的 10 个 timeBlock 毫秒长的块，而是从 timeBlock 的最近除数开始将其拆分为 11 个块。

要修复它，您应该首先计算delta = end - $time，然后使用它而不是原来的$time 来构建您的_id。

这是我的意思的一个例子：

Document.aggregate({
    $match: {
        time: {
            $gte: new Date(end - 10 * timeBlock),
            $lt: new Date(end)
        }
    }
}, {
    $project: {
        time: 1,
        delta: { $subtract: [
            new Date(end),
            "$time"
        ]}
    }
}, {
    $project: {
        time: 1,
        delta: { $subtract: [
            "$delta",
            { $mod: [
                "$delta",
                timeBlock
            ]}
        ]}
    }
}, {
    $group: {
        _id: { $subtract: [
            new Date(end),
            "$delta"
        ]},
        count: { $sum: 1 }
    }
}, {
    $project: {
        time: "$_id",
        count: 1,
        _id: 0
    }
}, {
    $sort: {
        time: 1
    }
}, function(err, result) {
    // ...
})

我还建议您使用原始时间值（以毫秒为单位），因为它更容易并且可以防止您出错。您可以使用$project 运算符在$group 之后将time 转换为timeParts。