【问题标题】:What's the SQL statement for returning multiple averages from a single table dynamically从单个表中动态返回多个平均值的 SQL 语句是什么
【发布时间】:2015-11-19 08:13:36
【问题描述】:

我需要根据用户设置生成动态结果,以减少通过帖子返回的数量。示例表:

+----------+-------+---------------------+
| cpu_name | used  | timestamp           |
+----------+-------+---------------------+
| CPU 3    | 0.200 | 2015-11-19 03:09:11 |
| CPU 2    | 0.000 | 2015-11-19 03:09:11 |
| CPU 1    | 0.000 | 2015-11-19 03:09:11 |
| CPU 0    | 0.025 | 2015-11-19 03:09:11 |
| CPU 3    | 0.000 | 2015-11-19 03:09:10 |
| CPU 2    | 0.000 | 2015-11-19 03:09:10 |
| CPU 1    | 0.000 | 2015-11-19 03:09:10 |
| CPU 0    | 0.000 | 2015-11-19 03:09:10 |
| CPU 3    | 0.000 | 2015-11-19 03:09:09 |
| CPU 2    | 0.000 | 2015-11-19 03:09:09 |
| CPU 1    | 0.000 | 2015-11-19 03:09:09 |
| CPU 0    | 0.122 | 2015-11-19 03:09:09 |
| CPU 3    | 0.000 | 2015-11-19 03:09:07 |
| CPU 2    | 0.225 | 2015-11-19 03:09:07 |
| CPU 1    | 0.000 | 2015-11-19 03:09:07 |
| CPU 0    | 0.000 | 2015-11-19 03:09:07 |
| CPU 0    | 0.025 | 2015-11-19 04:45:01 |
+----------+-------+---------------------+

需要对每个 cpu 的行进行平均,每 X 小时/天/等。

伪 SLQ(我如何用一条 SQL 语句做到这一点):

$time = 10
$unit = DAYS
$sample_factor = 1 //hour
for each CPU:
     $sql = "SELECT AVERAGE_every_hour(cpu_use) FROM tbl_cpu_use WHERE timestamp > (NOW() - INTERVAL ". $time. " ". $unit)"
     RETURN RESULTS BUT AS IF IT WERE ONE QUERY

例如 如果

$time = 1
$unit = Hour
$sample_factor = 1 //hour

结果是:

+----------+-------+---------------------+
| cpu_name | used  | timestamp           |
+----------+-------+---------------------+
| CPU 3    | 0.200 | 2015-11-19 03 |
| CPU 2    | 0.000 | 2015-11-19 03 |
| CPU 1    | 0.000 | 2015-11-19 03 |
| CPU 0    | 0.025 | 2015-11-19 03 |
| CPU 0    | 0.025 | 2015-11-19 05 |

但是如果

$time = 1
$unit = Hour
$sample_factor = .5 //hour

结果是

+----------+-------+---------------------+
| cpu_name | used  | timestamp           |
+----------+-------+---------------------+
| CPU 3    | 0.200 | 2015-11-19 03:00 |
| CPU 2    | 0.000 | 2015-11-19 03:00 |
| CPU 1    | 0.000 | 2015-11-19 03:00 |
| CPU 0    | 0.025 | 2015-11-19 03:00 |
| CPU 3    | 0.200 | 2015-11-19 03:30 |
| CPU 2    | 0.000 | 2015-11-19 03:30 |
| CPU 1    | 0.000 | 2015-11-19 03:30 |
| CPU 0    | 0.025 | 2015-11-19 03:30 |
| CPU 0    | 0.025 | 2015-11-19 05:00 |

注意:忽略结果中的“使用”列值,假设它们是一段时间内的平均值。 'timestamp' 列和平均是最重要的。

编辑: 在 mySQL 工作台中进行试验后,我认为我已经非常接近了(仍然难以确定准确性以及设置间隔的时间,但我认为这已经非常接近了简洁):

注意:在填充数据时添加了一个硬 unix 时间戳,每秒几乎没有额外的处理,这对这部分很有帮助。

SET @time_interval := date_sub(NOW(), INTERVAL 2 HOUR);
SET @sample_interval := 60;

SELECT cpu_name, AVG(used) as used, @sample_interval*AVG(ROUND(unix_timestamp/@sample_interval)) as unix_timestamp FROM 
    (SELECT cpu_name, used, @sample_interval*ROUND(unix_timestamp/@sample_interval) As unix_timestamp, `timestamp` FROM BH_DB.tbl_cpu_use WHERE `timestamp`>@time_interval ORDER BY id DESC LIMIT 18446744073709551615) AS sorted_table
GROUP BY cpu_name, unix_timestamp ORDER BY unix_timestamp;

【问题讨论】:

  • 如果我们将| CPU 0 | 0.025 | 2015-11-19 04:45 |添加到数据集中呢?
  • 这取决于采样因子是什么,但在最后一个屏幕截图中,它将被平均以创建一条线 | CPU 0 | 0.025 | 2015-11-19 04:30 |或 | CPU 0 | 0.025 | 2015-11-19 05:00 |;我认为哪一个并不重要,或者即使它是两条线的平均值......每秒一次 4 个 CPU 的 90 天日志是在图表上趋势的大量数据,所以我需要得到样本量减少,但准确和平均似乎是唯一的选择。我知道我可以通过许多查询和一个 PHP 循环来做到这一点,但我听说有一种在 SQL 中处理的超快速方式的传言。
  • 请相应地编辑您的问题
  • 我现在有。我想你可能会混淆上面的表格。实际的表永远不会得到 2015-11-19 04:45,它也总是得到秒数。我之前的观点是,在平均许多记录时,第 45 秒是向上还是向下取整并不重要,只要它是一致的。
  • 我敢肯定,我们至少有一个人很困惑

标签: mysql logging average sampling


【解决方案1】:

设置

create table tbl_cpu_use
(
  cpu_name varchar(10) not null,
  used decimal(5,4) not null,
  `timestamp` timestamp not null,
  primary key ( cpu_name, `timestamp` )
);

insert into tbl_cpu_use
( cpu_name, used, `timestamp` )
values
( 'CPU 3'    , 0.200 , '2015-11-19 03:39:11' ),
( 'CPU 2'    , 0.000 , '2015-11-19 03:39:11' ),
( 'CPU 1'    , 0.000 , '2015-11-19 03:39:11' ),
( 'CPU 0'    , 0.025 , '2015-11-19 03:39:11' ),
( 'CPU 3'    , 0.200 , '2015-11-19 03:09:11' ),
( 'CPU 2'    , 0.000 , '2015-11-19 03:09:11' ),
( 'CPU 1'    , 0.000 , '2015-11-19 03:09:11' ),
( 'CPU 0'    , 0.025 , '2015-11-19 03:09:11' ),
( 'CPU 3'    , 0.000 , '2015-11-19 03:09:10' ),
( 'CPU 2'    , 0.000 , '2015-11-19 03:09:10' ),
( 'CPU 1'    , 0.000 , '2015-11-19 03:09:10' ),
( 'CPU 0'    , 0.000 , '2015-11-19 03:09:10' ),
( 'CPU 3'    , 0.000 , '2015-11-19 03:09:09' ),
( 'CPU 2'    , 0.000 , '2015-11-19 03:09:09' ),
( 'CPU 1'    , 0.000 , '2015-11-19 03:09:09' ),
( 'CPU 0'    , 0.122 , '2015-11-19 03:09:09' ),
( 'CPU 3'    , 0.000 , '2015-11-19 03:09:07' ),
( 'CPU 2'    , 0.225 , '2015-11-19 03:09:07' ),
( 'CPU 1'    , 0.000 , '2015-11-19 03:09:07' ),
( 'CPU 0'    , 0.000 , '2015-11-19 03:09:07' )
;

create view digits
as
select 0 as num
union all
select 1
union all
select 2
union all
select 3
union all
select 4
union all
select 5
union all
select 6
union all
select 7
union all
select 8
union all
select 9
;

查询

-- define the sampling interval size
set @interval_seconds := 600;

select slots.cpu_name, 
-- for when cpu isnt active or no data for, use 0
avg(coalesce(cpu.used, 0)) as avg_used, 
slots.`time`
from 
(
-- construct consecutive timeslots starting from minimum timestamp
-- and definition of a decimal number as weighted sum of powers of 10
select `min` + interval (a2.num*100 + a1.num*10 + a0.num) * @interval_seconds second as `time`, cpu_names.cpu_name
from
-- get the minimum and maximum timestamp from tbl_cpu_use timeseries
(
  select max(`timestamp`) as `max`, min(`timestamp`) as `min`
  from tbl_cpu_use
) bounds
cross join
-- get all cpu_names to duplicate across timeslots when cpus arent active
(
  select distinct cpu_name
  from tbl_cpu_use
) cpu_names
cross join digits a2
cross join digits a1
cross join digits a0
-- filter timeslots between timeseries min and max
where `min` + interval (a2.num*100 + a1.num*10 + a0.num) * @interval_seconds second
<=     `max`
) slots
-- include also information for timeslots when cpus arent active
left join tbl_cpu_use cpu
on timestampdiff(second, slots.`time`, cpu.`timestamp`) between 0 and @interval_seconds
and slots.cpu_name = cpu.cpu_name
group by slots.cpu_name, slots.`time`
order by slots.`time`, slots.cpu_name
;

输出

+----------+------------+---------------------+
| cpu_name | avg_used   | time                |
+----------+------------+---------------------+
| CPU 0    | 0.03675000 | 2015-11-19 03:09:07 |
| CPU 1    | 0.00000000 | 2015-11-19 03:09:07 |
| CPU 2    | 0.05625000 | 2015-11-19 03:09:07 |
| CPU 3    | 0.05000000 | 2015-11-19 03:09:07 |
| CPU 0    | 0.00000000 | 2015-11-19 03:19:07 |
| CPU 1    | 0.00000000 | 2015-11-19 03:19:07 |
| CPU 2    | 0.00000000 | 2015-11-19 03:19:07 |
| CPU 3    | 0.00000000 | 2015-11-19 03:19:07 |
| CPU 0    | 0.00000000 | 2015-11-19 03:29:07 |
| CPU 1    | 0.00000000 | 2015-11-19 03:29:07 |
| CPU 2    | 0.00000000 | 2015-11-19 03:29:07 |
| CPU 3    | 0.00000000 | 2015-11-19 03:29:07 |
| CPU 0    | 0.02500000 | 2015-11-19 03:39:07 |
| CPU 1    | 0.00000000 | 2015-11-19 03:39:07 |
| CPU 2    | 0.00000000 | 2015-11-19 03:39:07 |
| CPU 3    | 0.20000000 | 2015-11-19 03:39:07 |
+----------+------------+---------------------+

sqlfiddle

【讨论】:

  • 哇,这是我多年来见过的最大的 SQL 语句。我理解了其中的 60%。将更多地研究并尝试实施;在此之前,我只想确认您是否认为这会比在 PHP 中实现所有这些逻辑更快。再次感谢,我的朋友,你掌握了 SQL 技能。
  • 它应该更快(性能方面)。但维护起来可能更复杂
  • 好吧,我会为努力而投票,即使 OP 不会
  • 我也是。。虽然经过几个小时的学习,我仍然不明白。不过它有帮助,给了我一个方向。请查看我上面的编辑,在练习和添加字段后,我可能非常接近 3 行解决方案。我们的想法是获取此查询,并使用 PHP 分配顶部变量以获得结果。这与您的解决方案相比效率如何?
  • 只要:在每个时间间隔内报告所有 cpu(中断、数据质量等),您的就可以工作。除非您拥有大量数据 - 基准测试并查看,否则这两者之间的性能差异可能无关紧要。不过,它们是不同的查询,上面针对可能的数据质量问题,例如在周期内对 cpu 的不完整报告等。如果您知道您的数据质量符合上述几点,则使用更简单的查询
【解决方案2】:
SELECT cpu_name, AVG(used) as used, unix_timestamp FROM 
     (SELECT cpu_name, used, (". $sql_sample_size. "*ROUND(unix_timestamp/". $sql_sample_size. ")) As unix_timestamp FROM tbl_cpu_use WHERE timestamp>(NOW() - INTERVAL ". $time. " ". $sql_unit. ")) AS sub_table 
GROUP BY cpu_name,unix_timestamp ORDER BY unix_timestamp,cpu_name DESC;

似乎产生了所需的输出:

cpu_name, used, unix_timestamp, unix_timestamp

CPU 1, 0.0420843, 1447966800.0000, 2015-11-19 16:29:59
CPU 3, 0.0248727, 1447966800.0000, 2015-11-19 16:29:59
CPU 0, 0.0728558, 1447966800.0000, 2015-11-19 16:29:59
CPU 2, 0.0388895, 1447966800.0000, 2015-11-19 16:29:59
CPU 2, 0.0405227, 1447970400.0000, 2015-11-19 17:29:59
CPU 1, 0.0445057, 1447970400.0000, 2015-11-19 17:29:59
CPU 3, 0.0288837, 1447970400.0000, 2015-11-19 17:29:59
CPU 0, 0.0663175, 1447970400.0000, 2015-11-19 17:29:59
CPU 1, 0.0522862, 1447974000.0000, 2015-11-19 18:14:48
CPU 3, 0.0358891, 1447974000.0000, 2015-11-19 18:14:48
CPU 0, 0.0551599, 1447974000.0000, 2015-11-19 18:14:48
CPU 2, 0.0378004, 1447974000.0000, 2015-11-19 18:14:48

【讨论】:

  • 接近了……图有点跳跃。
  • 我更正了,最后的 ORDER BY 是导入的,如果没有这两个顺序,它会产生相当随机的排序......现在图表不会跳到过去的值上。跨度>
猜你喜欢
  • 1970-01-01
  • 1970-01-01
  • 1970-01-01
  • 2013-11-29
  • 1970-01-01
  • 2016-03-20
  • 2014-10-31
  • 2021-12-20
  • 1970-01-01
相关资源
最近更新 更多