带有批次的 BigQuery ARRAY_AGG

Question

是否有一些简单的方法可以将值聚合成批次（带保存顺序）

从这里

key | value
----+------
a   |  1
a   |  2
a   |  3
a   |  4
a   |  5
a   |  6

b   |  9
b   |  8
b   |  7
b   |  6
b   |  5
b   |  4

到这里

key | batches
----+-------------------------------------------------------------
a   | [{'values': [1, 2]}, {'values': [3, 4]}, {'values': [5, 6]}]
b   | [{'values': [9, 8]}, {'values': [7, 6]}, {'values': [5, 4]}]

{'values': [1, 2]} - 结构
[{'values': [1, 2]}, {'values': [3, 4]}, {'values': [5, 6]}] - 重复结构

Answer 1

对于 BigQuery 标准 SQL

是否有一些简单的方法可以将值聚合成批次（带有保存顺序）

注意；为了“保存订单”-您需要在数据中显示此订单-我假设您有一些定义该订单的东西（我为此使用ts字段）-通常是时间戳或订单/位置数据类型等。

所以，下面是实现上述目标的起点

#standardSQL
SELECT key, 
  ARRAY_AGG(value ORDER BY pos) AS batch,
  DIV(pos - 1, 2) batch_num
FROM (
  SELECT *, 
    ROW_NUMBER() OVER(PARTITION BY key ORDER BY ts) pos,
    DIV(ROW_NUMBER() OVER(PARTITION BY key ORDER BY ts) - 1, 2) batch
  FROM `project.dataset.table`
)
GROUP BY key, batch_num
   

如果应用于您问题中的样本数据 - 输出是

从这里，您可以将上述结果打包成您需要的任何最终格式 - 例如

#standardSQL
SELECT key, ARRAY_AGG(batch ORDER BY batch_num) batches
FROM (
  SELECT key, 
    STRUCT(ARRAY_AGG(value ORDER BY pos) AS values) batch,
    DIV(pos - 1, 2) batch_num
  FROM (
    SELECT *, 
      ROW_NUMBER() OVER(PARTITION BY key ORDER BY ts) pos,
      DIV(ROW_NUMBER() OVER(PARTITION BY key ORDER BY ts) - 1, 2) batch
    FROM `project.dataset.table`
  )
  GROUP BY key, batch_num
)
GROUP BY key

给出以下输出（非常接近您问题中的预期示例）

[
  {
    "key": "a",
    "batches": [
      {
        "values": [
          "1",
          "2"
        ]
      },
      {
        "values": [
          "3",
          "4"
        ]
      },
      {
        "values": [
          "5",
          "6"
        ]
      }
    ]
  },
  {
    "key": "b",
    "batches": [
      {
        "values": [
          "9",
          "8"
        ]
      },
      {
        "values": [
          "7",
          "6"
        ]
      },
      {
        "values": [
          "5",
          "4"
        ]
      }
    ]
  }
]