【问题标题】:Lateral Flatten Snowflake from a Variant table变体表中的横向展平雪花
【发布时间】:2020-06-17 14:45:43
【问题描述】:

我有一个名为 raw_json 的变体表,其中包含多个 json 文件,这些文件对 ID 是唯一的,但在每个 json 中包含相似的数据点。它们位于 jsontext 列中。这是上下文的SS。我正在尝试将raw_json 表的每一行展平为普通的表视图。两个数组索引需要对齐才能分配正确的值。

以下是raw_json 表中的两行以及 json 的结构。

{
 "ID": "PO-103",
 "content": {
   "EEList": [
     {
       "EEListID": "PO-103-1",
       "EEProductID": "XXX1976",

     },
     {
       "EEListID": "PO-103-2",
       "EEProductID": "XXX1977",
     },
     {
       "EEListID": "PO-103-3",
       "EEProductID": "XXX1978",
     }
   ],
   "EENotesList": [
     {
       "FirstName": "John",
       "LastName": "Smith",
       "pxObjClass": "XX-XXSales-Work-XX"
     },
     {
       "FirstName": "Bob",
       "LastName": "Joe",
       "pxObjClass": "XX-XXSales-Work-XX"
     },
     {
       "FirstName": "Mike",
       "LastName": "Smith",
       "pxObjClass": "XX-XXSales-Work-XX"
     }
   ],
 }
}
{
  "ID": "PO-104",
  "content": {
    "EEList": [
      {
        "EEListID": "PO-104-1",
        "EEProductID": "XXX1979",

      },
      {
        "EEListID": "PO-104-2",
        "EEProductID": "XXX1980",
      },
      {
        "EEListID": "PO-104-3",
        "EEProductID": "XXX1981",
      }
    ],
    "EENotesList": [
      {
        "FirstName": "Sarah",
        "LastName": "Butler",
        "pxObjClass": "XX-XXSales-Work-XX"
      },
      {
        "FirstName": "Jessica",
        "LastName": "Adams",
        "pxObjClass": "XX-XXSales-Work-XX"
      }
    ],
  }
}

放入这样的表格中(我需要这个)

+--------+----------+-------------+-----------+----------+-------------------+
|   ID   | EEListID | EEProductID | FirstName | LastName |    pxObjClass     |
+--------+----------+-------------+-----------+----------+-------------------+
| PO-103 | PO-103-1 | XXX1976     | John      | Smith    | X-XXSales-Work-XX |
| PO-103 | PO-103-2 | XXX1977     | Bob       | Joe      | X-XXSales-Work-XX |
| PO-103 | PO-103-3 | XXX1978     | Mike      | Smith    | X-XXSales-Work-XX |
| PO-104 | PO-104-1 | XXX1979     | Sarah     | Butler   | X-XXSales-Work-XX |
| PO-104 | PO-104-2 | XXX1980     | Jessica   | Adams    | X-XXSales-Work-XX |
+--------+----------+-------------+-----------+----------+-------------------+

我已经能够将 EENoteList 数组展平为一个表并将正确的 ID 分配给该表。到目前为止,这是我的代码:在不展开表格的情况下添加 EEList 值是我出错的地方。

select
    jsontext:ID::varchar as ID,
    en.value:FirstName::varchar as FirstName,
    en.value:LastName::varchar as LastName,
    en.value:pxObjClass::varchar as pxObjClass
   -- concat(ID, EEProductID, FirstName, LastName)

    from raw_json,
    lateral flatten (input => jsontext:content:EENotesList, outer => false) en;

产生这张表(我有这个)

+--------+-----------+----------+-------------------+
|   ID   | FirstName | LastName |    pxObjClass     |
+--------+-----------+----------+-------------------+
| PO-103 | John      | Smith    | X-XXSales-Work-XX |
| PO-103 | Bob       | Joe      | X-XXSales-Work-XX |
| PO-103 | Mike      | Smith    | X-XXSales-Work-XX |
| PO-104 | Sarah     | Butler   | X-XXSales-Work-XX |
| PO-104 | Jessica   | Adams    | X-XXSales-Work-XX |
| PO-104 | Terrence  | Williams | X-XXSales-Work-XX |
+--------+-----------+----------+-------------------+

【问题讨论】:

    标签: snowflake-cloud-data-platform flatten snowflake-schema lateral


    【解决方案1】:

    所以这个答案大多是将数据放入 CTE 的序言,但是“iff”两个数组的顺序是同步的,您可以只使用 flatten 的索引来访问另一种类型的原始数组:

    WITH raw_json AS (
    select PARSE_json(column1) AS jsontext FROM VALUES 
     ('{
     "ID": "PO-103",
     "content": {
       "EEList": [
         {
           "EEListID": "PO-103-1",
           "EEProductID": "XXX1976",
    
         },
         {
           "EEListID": "PO-103-2",
           "EEProductID": "XXX1977",
         },
         {
           "EEListID": "PO-103-3",
           "EEProductID": "XXX1978",
         }
       ],
       "EENotesList": [
         {
           "FirstName": "John",
           "LastName": "Smith",
           "pxObjClass": "XX-XXSales-Work-XX"
         },
         {
           "FirstName": "Bob",
           "LastName": "Joe",
           "pxObjClass": "XX-XXSales-Work-XX"
         },
         {
           "FirstName": "Mike",
           "LastName": "Smith",
           "pxObjClass": "XX-XXSales-Work-XX"
         }
       ],
     }
    }'), ('{
      "ID": "PO-104",
      "content": {
        "EEList": [
          {
            "EEListID": "PO-104-1",
            "EEProductID": "XXX1979",
    
          },
          {
            "EEListID": "PO-104-2",
            "EEProductID": "XXX1980",
          },
          {
            "EEListID": "PO-104-3",
            "EEProductID": "XXX1981",
          }
        ],
        "EENotesList": [
          {
            "FirstName": "Sarah",
            "LastName": "Butler",
            "pxObjClass": "XX-XXSales-Work-XX"
          },
          {
            "FirstName": "Jessica",
            "LastName": "Adams",
            "pxObjClass": "XX-XXSales-Work-XX"
          }
        ],
      }
    }') 
    )
    select
        jsontext:ID::varchar as ID,
        en.value:FirstName::varchar as FirstName,
        en.value:LastName::varchar as LastName,
        en.value:pxObjClass::varchar as pxObjClass,
        jsontext:content.EEList[en.index].EEListID::text as EEListID,
        jsontext:content.EEList[en.index].EEProductID::text as EEProductID
        from raw_json,
        lateral flatten (input => jsontext:content:EENotesList, outer => false) en;
    

    这会导致:

    ID  FIRSTNAME   LASTNAME    PXOBJCLASS  EELISTID    EEPRODUCTID
    PO-103  John    Smith   XX-XXSales-Work-XX  PO-103-1    XXX1976
    PO-103  Bob     Joe     XX-XXSales-Work-XX  PO-103-2    XXX1977
    PO-103  Mike    Smith   XX-XXSales-Work-XX  PO-103-3    XXX1978
    PO-104  Sarah   Butler  XX-XXSales-Work-XX  PO-104-1    XXX1979
    PO-104  Jessica Adams   XX-XXSales-Work-XX  PO-104-2    XXX1980
    

    【讨论】:

    • 这正是我想要的!您在哪里找到了如何使用 EEList[en.index]?在雪花文档中找不到任何索引..
    • 您能否扩展答案的第一部分?我从未使用过 CTE,因此缺乏上下文。
    • 只是假设它会工作一段时间,因为它是 JSON 对象上的 Javascript 访问样式。我很确定也有一个 get_member 之类的访问器。我只是使用.: 来访问子元素。
    • CTE(公用表表达式)docs.snowflake.net/manuals/user-guide/queries-cte.html 是一个表达式,它使命名表就像您可以在其他地方访问的东西一样,很像带有别名的子选择,但如果您加入该子表达式多次,它不需要定义多次。我经常使用然后你在逻辑上分离处理步骤,因为优化器可以并且确实将它全部混合起来,就好像它是子选择一样......
    猜你喜欢
    • 2021-08-16
    • 1970-01-01
    • 2021-04-25
    • 2019-08-10
    • 2016-08-16
    • 2020-03-24
    • 2020-02-09
    • 2022-08-20
    • 2022-01-21
    相关资源
    最近更新 更多