如何查询非规范化 BigQuery 表以输出嵌套和重复的字段

Question

我有一些数据定期加载到 BigQuery 数据库中。每一行代表作为履行订单的一部分的一件库存的移动。

相关订单和“order_product”信息作为嵌套记录嵌入在行中。

以下是一些行的示例：

|----------|----------|----------|------------------|------------------------|---------------------|---------------------------------|
| move_id  | quantity | order.id | order_product.id | order_product.quantity | order_product.price |      item_reference_number      |
|----------|----------|----------|------------------|------------------------|---------------------|---------------------------------|
|    1     |    1     |     1    |         1        |            1           |           5         |               ABC               |
|----------|----------|----------|------------------|------------------------|---------------------|---------------------------------|
|    2     |    1     |     1    |         2        |            1           |           7         |               DEF               |
|----------|----------|----------|------------------|------------------------|---------------------|---------------------------------|
|    3     |    1     |     1    |         2        |            1           |           7         |               XYZ               |
|----------|----------|----------|------------------|------------------------|---------------------|---------------------------------|

如您所见，该表显示了三个股票走势。都与订单 1 相关。

Order 1 由一个 order_product 1 和一个 order_product 2 组成。Order_product 1 由一个库存移动组成，move_id 1 用于项目 ABC。

Order_product 2 包含两个库存变动，move_id 2 用于商品 DEF，move_id 3 用于商品 XYZ。

如何编写查询以将此数据转换为具有适当嵌套/重复字段的表？换句话说，我希望数据看起来像这样：

|-----------|------------------|------------------------|---------------------|------------------------|---------------------|----------------------------------|
| order_id  | order_product.id | order_product.quantity | order_product.price |      stock_move.id     | stock_move.quantity | stock_move.item_reference_number |
|-----------|------------------|------------------------|---------------------|------------------------|---------------------|----------------------------------|
|     1     |         1        |            1           |          5          |            1           |           1         |                ABC               |
|           |------------------|------------------------|---------------------|------------------------|---------------------|----------------------------------|
|           |         2        |            1           |          7          |            2           |           1         |                DEF               |
|           |                  |                        |                     |------------------------|---------------------|----------------------------------|
|           |                  |                        |                     |            3           |           1         |                XYZ               |
|-----------|------------------|------------------------|---------------------|------------------------|---------------------|----------------------------------|

我一直在阅读this post，这似乎表明ARRAY_AGG 可能是我需要的，但我不知道如何正确使用它来解决我的问题。

我认为我的问题是我正在努力将嵌套的 order_products 减少到每一行，同时为每个 order_product 填充正确的嵌套/重复库存移动。

甚至可以按照我的要求做吗？我非常感谢任何帮助我指出正确的方向。

Answer 1

以下是 BigQuery 标准 SQL

#standardSQL
SELECT order_id,
  ARRAY_AGG(product ORDER BY product.id) order_product,
  ARRAY_CONCAT_AGG(stock_move) stock_move
FROM (
  SELECT order_id, 
    STRUCT(order_product.id, order_product.quantity, order_product.price) product,
    ARRAY_AGG(STRUCT(move_id AS id, quantity AS quantity, item_reference_number AS item_reference_number)) stock_move
  FROM `project.dataset.table`
  GROUP BY order_id, order_product.id, order_product.quantity, order_product.price
  ORDER BY order_product.id -- <-- this is to make sure stock_move array is ordered as in your expected output  - but really not needed here
)
GROUP BY order_id

当应用于您的示例数据时 - 上面会产生如下结果

我不确定这是否正是你的意思，因为你的例子仍然有点模棱两可，但希望这能给你一个想法

另外请注意：我假设您的示例中的 order.id 实际上是 order_id 否则它没有多大意义，但我可能对此有误（正如我提到的那样，您的示例仍然“有点”模棱两可）

Answer 2

以下 SQL 会满足您的期望吗？

我以 order_product 方式创建了 stock_move。

WITH original_table AS (
    SELECT 1 AS move_id, 1 AS quantity, STRUCT(1 AS id) AS `order`, STRUCT(1 AS id, 1 AS quantity, 5 AS price) AS order_product, "ABC" AS item_reference_number
    UNION ALL
    SELECT 2 AS move_id, 1 AS quantity, STRUCT(1 AS id) AS `order`, STRUCT(2 AS id, 1 AS quantity, 7 AS price) AS order_product, "DEF" AS item_reference_number
    UNION ALL
    SELECT 3 AS move_id, 1 AS quantity, STRUCT(1 AS id) AS `order`, STRUCT(2 AS id, 1 AS quantity, 7 AS price) AS order_product, "XYZ" AS item_reference_number
),

t1 AS (
    SELECT DISTINCT
        move_id,
        quantity,
        `order`.id AS order_id,
        order_product.id AS order_product_id,
        order_product.quantity AS order_product_quantity,
        order_product.price AS order_product_price,
        item_reference_number
    FROM original_table
),

t2 AS (
    SELECT
        order_id,
        order_product_id,
        order_product_quantity,
        order_product_price,
        ARRAY_AGG(STRUCT(move_id, quantity, item_reference_number) ORDER BY move_id) AS stock_move
    FROM t1
    GROUP BY order_id, order_product_id, order_product_quantity, order_product_price
),

t3 AS (
    SELECT
        order_id,
        ARRAY_AGG(STRUCT(order_product_id AS id, order_product_quantity AS quantity, order_product_price AS price, stock_move) ORDER BY order_product_id) AS order_product
    FROM t2
    GROUP BY order_id
)

SELECT * FROM t3

|-----------|------------------|------------------------|---------------------|----------------------------------|-----------------------------------|------------------------------------------------|
| order_id  | order_product.id | order_product.quantity | order_product.price | order_product.stock_move.move_id | order_product.stock_move.quantity | order_product.stock_move.item_reference_number |
|-----------|------------------|------------------------|---------------------|----------------------------------|-----------------------------------|------------------------------------------------|
|     1     |         1        |            1           |          5          |              1                   |           1                       |                ABC                             |
|           |------------------|------------------------|---------------------|----------------------------------|-----------------------------------|------------------------------------------------|
|           |         2        |            1           |          7          |              2                   |           1                       |                DEF                             |
|           |                  |                        |                     |----------------------------------|-----------------------------------|------------------------------------------------|
|           |                  |                        |                     |              3                   |           1                       |                XYZ                             |
|-----------|------------------|------------------------|---------------------|----------------------------------|-----------------------------------|------------------------------------------------|