如何取消嵌套 BigQuery sql 中的嵌套重复记录

Question

我正在尝试有条件地检索 BigQuery 标准 sql 中的某些字段。问题是我正在使用的表有多个级别的嵌套数据。我使用连续 CROSS JOIN 的策略（参考以前的 CROSS JOIN 来更深入地了解嵌套数组）似乎没有按预期工作，因为我不断收到 GCP 的错误：Table name "nhht" missing dataset while no default dataset is set in the request. 请参阅下面的表格和预期（和解释）输出以及我尝试的查询。提前致谢：

尝试查询：

select tab2.house_type_id as `house_type_id`,
case when fmd.house_type_metadata_secure.house_type_id=tab2.house_type_id then fh.house_color_data.house_door_color else tab2.house_door_color end as `house_door_color`,
tab2.house_name,
tab2.house_roof_color
from `table1` as tab1,
unnest(tab1.favorite_houses) as fh,
unnest(fh.house_info.house_type_data.house_type_metadata) as fmd,
unnest(tab1.newhouses) as nh,
unnest(tab1.oldhouses) as oh,
unnest(nh.house_type) as nhht,
unnest(oh.house_type) as ohht,
(select * from nhht union all select * from ohht) as comb
left join `table2` as tab2
on comb.house_type_info.house_type_id=tab2.house_type_id

带有解释的期望输出：

逻辑是查看table1 中new_houses 和old_houses 中的所有house_type_id。 （请注意，我们保证出现在我们数据中所有 new_houses 和 old_houses 数组中的所有 house_type_id 的集合不包含重复项，并且 old_houses 中的 house_type_id 也可能只出现在一个 favorite_houses 列表中。我们还保证如果 house_type_id 出现在任何地方在table1中它必须出现在table2中，但是table2中可能有一些house_type_id没有出现在table1中。）

• 我们将获取这些 house_type_id 中的每一个，并从 table2 中选择其关联名称
• 如果此 house_type_id 出现在 favorite_houses 中，则从 favourite_houses 中的 table1 中选择它的 house_door_color，如果没有从 table2 中获取它
• 从 table2 中选择其 house_roof_color

{
    {
        "house_type_id": "hid2000", --how we identify data between tables
        "house_name": "oak st.", --should come from table 2
        "house_door_color": "red", --should come from table 1 if house_type_id in favorite houses
        "house_roof_color": "purple", --should come from table 2
    },
    {
        "house_type_id": "hid1000"
        "house_name": "elm st."
        "house_door_color": "black"
        "house_roof_color": "black"
    },
    {
        "house_type_id": "hid3000",
        "house_name": "juniper st.",
        "house_door_color": "grey",
        "house_roof_color": "grey"
    }
}

table1：

{
    "name": "tom",
    "new_houses": [
        {
            "house_name": "elm st.",
            "house_type": [
                {
                    "house_color": "green",
                    "house_type_info": {
                        "house_type_id": "hid1000"
                    }
                }
            ]
        }
    ],
    "old_houses": [
        {
            "house_name": "oak st.",
            "house_type": [
                {
                    "house_color": "blue",
                    "house_type_info": {
                        "house_type_id": "hid2000"
                    }
                }
            ]
        }
    ],
    "favorite_houses": [
        {
            "house_info": {
                "house_name":"oak st.",
                "house_type_data": {
                    "house_type_metadata": [
                        {
                            "house_type_metadata_secure": {
                                "house_type_id": "hid2000"
                            }
                        }
                    ]
                }
            },
            "house_color_data": {
                "house_door_color": "red",
                "house_color": "blue"
            }
        }
    ]
}

table2：

    {
        "house_type_id": "hid2000",
        "house_door_color": "yellow",
        "house_roof_color": "purple",
        "house_name": "oak st."
    },
    {
        "house_type_id": "hid1000",
        "house_door_color": "black",
        "house_roof_color": "black",
        "house_name": "elm st."
    },
    {
        "house_type_id": "hid3000",
        "house_door_color": "grey",
        "house_roof_color": "grey",
        "house_name": "juniper st."
    }

Answer 1

试试这个

with all_house_type_ids
as
(
select  ht.house_type_id as house_type_id,
    fh.house_color_data.house_door_color as house_door_color
from    table1,
    unnest(new_houses) as nh,
    unnest(nh.house_type) as ht
    unnest(favorite_houses) as fh,
inner join
    unnest(fh.house_info.house_type_metadata) as htmd
on ht.house_type_info.house_type_id = htmd.house_type_id

UNION ALL

select  ht.house_type_id as house_type_id,
    fh.house_color_data.house_door_color as house_door_color
from    table1,
    unnest(old_houses) as oh,
    unnest(oh.house_type) as ht
    unnest(favorite_houses) as fh,
inner join
    unnest(fh.house_info.house_type_metadata) as htmd
on ht.house_type_info.house_type_id = htmd.house_type_id
),
select 
    ahtpi.house_type_id,
    table2.house_name,
    ahtpi.house_door_color,
    table2.house_roof_color

from all_house_type_ids ahtpi
left join table2 
using(house_type_id)