我可以查询clickhouse中累积列的每小时增量吗？

Question

我想节省活动时间和每 30 秒的总发电量。总金额不会每次都重置为零。这只是从计量表开始到现在的总数，而不是 30 秒内产生的总量。

有没有什么方法可以查询每天、每周或每月的总发电量（可能不只是 sum 或 avg）？

或者设计一个AggregatingMergeTree 表？

我不需要保留每条记录，只需要每天、每周和每月的汇总。

例如：

create table meter_record (
   event_time Datetime,
   generated_total Int64
)

Answer 1

更新

对于 median、avg、min、maxSimpleAggregateFunction 而不是 AggregateFunction /em> 加快聚合计算。

---

我们建议您需要为此表计算 median、average 和 dispersion 聚合：

CREATE TABLE meter_record (
   event_time Datetime,
   generated_total Int64  
)
ENGINE = MergeTree
PARTITION BY (toYYYYMM(event_time))
ORDER BY (event_time);

使用AggregatingMergeTree 计算所需的聚合：

CREATE MATERIALIZED VIEW meter_aggregates_mv
ENGINE = AggregatingMergeTree()
PARTITION BY toYYYYMM(day)
ORDER BY (day) 
AS
SELECT  
  toDate(toStartOfDay(event_time)) AS day, 
  /* aggregates to calculate the day's section left and right endpoints */
  minState(generated_total) min_generated_total,
  maxState(generated_total) max_generated_total,
  /* specific aggregates */
  medianState(generated_total) AS totalMedian,
  avgState(generated_total) AS totalAvg,
  varPopState(generated_total) AS totalDispersion
  /* ... */
FROM meter_record
GROUP BY day;

要获得所需的每日/每周/每月（以及任何以日为基础的聚合，如 Quarterly 或 yearly）聚合，请使用以下查询：

/* daily report */
SELECT 
  day,
  minMerge(min_generated_total) min_generated_total,
  maxMerge(max_generated_total) max_generated_total,
  medianMerge(totalMedian) AS totalMedian,
  avgMerge(totalAvg) AS totalAvg,
  varPopMerge(totalDispersion) AS totalDispersion
FROM meter_aggregates_mv
/*WHERE day >= '2019-02-05' and day < '2019-07-01'*/
GROUP BY day;

/* weekly report */
SELECT 
  toStartOfWeek(day, 1) monday,
  minMerge(min_generated_total) min_generated_total,
  maxMerge(max_generated_total) max_generated_total,
  medianMerge(totalMedian) AS totalMedian,
  avgMerge(totalAvg) AS totalAvg,
  varPopMerge(totalDispersion) AS totalDispersion
FROM meter_aggregates_mv
/*WHERE day >= '2019-02-05' and day < '2019-07-01'*/
GROUP BY monday;

/* monthly report */
SELECT 
  toStartOfMonth(day) month,
  minMerge(min_generated_total) min_generated_total,
  maxMerge(max_generated_total) max_generated_total,
  medianMerge(totalMedian) AS totalMedian,
  avgMerge(totalAvg) AS totalAvg,
  varPopMerge(totalDispersion) AS totalDispersion
FROM meter_aggregates_mv
/*WHERE day >= '2019-02-05' and day < '2019-07-01'*/
GROUP BY month;

/* get daily / weekly / monthly reports in one query (thanks @Denis Zhuravlev for advise) */
SELECT
  day,
  toStartOfWeek(day, 1) AS week,
  toStartOfMonth(day) AS month,
  minMerge(min_generated_total) min_generated_total,
  maxMerge(max_generated_total) max_generated_total,
  medianMerge(totalMedian) AS totalMedian,
  avgMerge(totalAvg) AS totalAvg,
  varPopMerge(totalDispersion) AS totalDispersion
FROM meter_aggregates_mv
/*WHERE (day >= '2019-05-01') AND (day < '2019-06-01')*/
GROUP BY month, week, day WITH ROLLUP
ORDER BY day, week, month;

备注：

• 你指出你不需要原始数据只是聚合，所以你可以将 meter_record-table 的引擎设置为Null，手动清理 meter_record（请参阅DROP PARTITION）或定义TTL 以自动执行此操作

• 删除原始数据是不好的做法，因为它无法计算历史数据的新聚合或恢复现有聚合等

• 物化视图meter_aggregates_mv 将只包含创建视图后插入到表meter_record 中的数据。要更改此行为，请在视图定义中使用 POPULATE