如何有效地计算 Google BigQuery 中数字序列的中位数？答案

【问题标题】：How to calculate median of a numeric sequence in Google BigQuery efficiently?如何有效地计算 Google BigQuery 中数字序列的中位数？
【发布时间】：2015-05-19 12:11:14
【问题描述】：

我需要有效地计算 Google BigQuery 中数字序列的中值。有可能吗？

【问题讨论】：

您的费率很低。对 SO 很重要，您必须使用已发布答案左侧、投票下方的勾号来标记已接受的答案。这将提高您的费率。通过访问此链接了解其工作原理：meta.stackoverflow.com/questions/5234/…
另见stackoverflow.com/questions/51981353/…

标签： google-bigquery median

【解决方案1】：

是的，可以使用 PERCENTILE_CONT 窗口函数。

返回基于线性插值的值组的值，在按照 ORDER BY 子句排序之后。

必须介于 0 和 1 之间。

此窗口函数需要在 OVER 子句中使用 ORDER BY。

所以一个示例查询就像（max() 只是为了跨组工作，但它不被用作数学逻辑，不应该让你感到困惑）

SELECT room,
      max(median) FROM   (SELECT room,
         percentile_cont(0.5) OVER (PARTITION BY room
                                    ORDER BY temperature) AS median    FROM
    (SELECT 1 AS room,
            11 AS temperature),
    (SELECT 1 AS room,
            12 AS temperature),
    (SELECT 1 AS room,
            14 AS temperature),
    (SELECT 1 AS room,
            19 AS temperature),
    (SELECT 1 AS room,
            13 AS temperature),
    (SELECT 2 AS room,
            20 AS temperature),
    (SELECT 2 AS room,
            21 AS temperature),
    (SELECT 2 AS room,
            29 AS temperature),
    (SELECT 3 AS room,
            30 AS temperature)) GROUP BY room

这会返回：

+------+-------------+
| room | temperature |
+------+-------------+
|    1 |          13 |
|    2 |          21 |
|    3 |          30 |
+------+-------------+

【讨论】：

能否请您提供更清晰简洁的查询？我无法理解以上内容。
@ManishAgrawal 尝试分段运行，你最终会明白，这个查询很简单。可能对您来说新的是 OVER() 东西，您需要进一步阅读，它是窗口函数的基础。如果 from 子句让您感到困惑，我尝试复制表结果，以便您可以复制粘贴并按原样运行此查询。
房间 1 的值 11,12,14,19,13 不应该是中位数 14？
@AndresUrregoAngel 中值来自有序的值范围。当然，按照您在上面描述的顺序，值“14”在中间。但是，这不是一个有序列表。有序列表将是11, 12, 13, 14, 19。因此，“13”是正确的中值

【解决方案2】：

替代解决方案，当您不需要绝对精确的结果并且近似值很好时 - 您可以使用 NTH 和 QUANTILES 聚合函数的组合。这种方法的优点是它比解析窗函数的可扩展性强得多，但缺点是它给出了近似的结果。

SELECT room,
       NTH(50, QUANTILES(temperature, 101)) FROM
    (SELECT 1 AS room,
            11 AS temperature),
    (SELECT 1 AS room,
            12 AS temperature),
    (SELECT 1 AS room,
            14 AS temperature),
    (SELECT 1 AS room,
            19 AS temperature),
    (SELECT 1 AS room,
            13 AS temperature),
    (SELECT 2 AS room,
            20 AS temperature),
    (SELECT 2 AS room,
            21 AS temperature),
    (SELECT 2 AS room,
            29 AS temperature),
    (SELECT 3 AS room,
            30 AS temperature) GROUP BY room

room temperature 
1    13  
2    21  
3    30

【讨论】：

我认为您需要 NTH(51, QUANTILES(temperature, 101)) 作为中位数，因为 NTH 是从 1 开始的。见cloud.google.com/bigquery/query-reference#quantiles

【解决方案3】：

2018 年更新包含更多指标：

BigQuery SQL: Average, geometric mean, remove outliers, median

出于我自己的记忆目的，使用出租车数据进行查询：

近似分位数：

SELECT MONTH(pickup_datetime) month, NTH(51, QUANTILES(tip_amount,101)) median
FROM [nyc-tlc:green.trips_2015]
WHERE tip_amount > 0
GROUP BY 1
ORDER BY 1

给出与 PERCENTILE_DISC 相同的结果：

SELECT month, FIRST(median) median
FROM (
  SELECT MONTH(pickup_datetime) month, tip_amount, PERCENTILE_DISC(0.5) OVER(PARTITION BY month ORDER BY tip_amount) median
  FROM [nyc-tlc:green.trips_2015]
  WHERE tip_amount > 0
)
GROUP BY 1
ORDER BY 1

标准 SQL：

#StandardSQL
SELECT DATE_TRUNC(DATE(pickup_datetime), MONTH) month, APPROX_QUANTILES(tip_amount,1000)[OFFSET(500)] median
FROM `nyc-tlc.green.trips_2015`
WHERE tip_amount > 0
GROUP BY 1
ORDER BY 1

【讨论】：