将字符串列表转换为表格答案

【问题标题】：Convert list of strings to table将字符串列表转换为表格
【发布时间】：2018-07-04 15:31:30
【问题描述】：

我只是一个sql初学者，想把字符串转成表格

申请日期：01/02/2018 请求：购买书籍联系方式：电子邮件：hi@gmail.com 电话：0123456789 订购查询：订单 ID：12345678 BL：87654321 产品：123456 书籍

申请日期：01/04/2018 请求：Retour table 联系人：Rodion Raskólnikov 电子邮件：hello@outlook.com 电话：9876543210 订购查询：订单 ID：87654321 BL：12345678 产品：654321 Tables

像这样：

我试过这个：

WITH raw_messages AS (SELECT lines
  FROM `my_table` 
  WHERE REGEXP_CONTAINS(lines, '^Date of application: '))

SELECT 
  REGEXP_EXTRACT(lines, r'^Date of application: [0-9]{2}/[0-9]{2}/[0-9]{4}') as date

FROM raw_messages

它没有按我的意愿工作，我不知道如何继续构建我的表。

【问题讨论】：

标签： sql google-bigquery

【解决方案1】：

如果您的字符串中的字段顺序未知/保证，但您知道其中的所有字段 - 下面足够聪明，可以正确解析这些字段

#standardSQL
WITH raw_messages AS (
  SELECT lines FROM `project.dataset.my_table` 
  WHERE REGEXP_CONTAINS(lines, '^Date of application: ')
), fields AS (
  SELECT 'Date of application' field, 'date' column UNION ALL
  SELECT 'Request', 'request' UNION ALL
  SELECT 'Contact', 'contact' UNION ALL
  SELECT 'email', 'email' UNION ALL
  SELECT 'Tel', 'phone' UNION ALL
  SELECT 'Order ID', 'id' UNION ALL
  SELECT 'BL', 'bl' UNION ALL
  SELECT 'Product', 'product' UNION ALL
  SELECT 'Ordered inquiry', '' UNION ALL
  SELECT 'Boundary of string', ''
), patterns AS ( 
  SELECT f1.field, f1. column, CONCAT(r'(?i) ',f1.field,': (.*)',f2.field,': ') pattern
  FROM fields f1 CROSS JOIN fields f2
), splits AS (SELECT ARRAY(
      SELECT AS STRUCT column, ARRAY_AGG(value ORDER BY LENGTH(value) LIMIT 1)[OFFSET(0)] value
      FROM (SELECT column, REGEXP_EXTRACT(CONCAT(' Boundary of string: ', lines, ' Boundary of string: '), pattern) value
        FROM patterns ) 
      WHERE NOT value IS NULL AND NOT column = '' GROUP BY column 
    ) arr FROM raw_messages
) SELECT 
  (SELECT value FROM UNNEST(arr) WHERE column='date')     AS DATE,
  (SELECT value FROM UNNEST(arr) WHERE column='request')  AS request,
  (SELECT value FROM UNNEST(arr) WHERE column='contact')  AS contact,
  (SELECT value FROM UNNEST(arr) WHERE column='email')    AS email,
  (SELECT value FROM UNNEST(arr) WHERE column='phone')    AS phone,
  (SELECT value FROM UNNEST(arr) WHERE column='id')       AS id,
  (SELECT value FROM UNNEST(arr) WHERE column='bl')       AS bl,
  (SELECT value FROM UNNEST(arr) WHERE column='product')  AS product
FROM splits

您可以使用与我的另一个答案相同的虚拟数据进行上述测试，显然结果应该是相同的

注意：如您所见 - 您需要明确设置 fields AS (...) CTE，字符串中的所有字段和相应的列名以任何顺序使用但很重要 - 您需要在此处再添加一个条目 - 'Boundary of string'

【讨论】：

【解决方案2】：

以下是 BigQuery 标准 SQL

假设您的行中字段的顺序设置为您的示例中的设置

#standardSQL
WITH raw_messages AS (
  SELECT lines
  FROM `my_table` 
  WHERE REGEXP_CONTAINS(lines, '^Date of application: ')
)
SELECT 
  REGEXP_EXTRACT(lines, r'(?i)^Date of application: ([0-9]{2}/[0-9]{2}/[0-9]{4})') AS DATE,
  REGEXP_EXTRACT(lines, r'(?i) Request: (.*?) Contact: ') AS request,
  REGEXP_EXTRACT(lines, r'(?i) Contact: (.*?) email: ') AS contact,
  REGEXP_EXTRACT(lines, r'(?i) email: (.*?) Tel: ') AS email,
  REGEXP_EXTRACT(lines, r'(?i) Tel: (.*?) Ordered inquiry: ') AS phone,
  REGEXP_EXTRACT(lines, r'(?i) Order ID: (.*?) BL: ') AS id,
  REGEXP_EXTRACT(lines, r'(?i) BL: (.*?) Product: ') AS bl,
  REGEXP_EXTRACT(lines, r'(?i) Product: (.*?)$') AS product
FROM raw_messages

您可以使用您问题中的虚拟数据进行测试，如下所示

#standardSQL
WITH `project.dataset.my_table` AS (
  SELECT 'Date of application: 01/02/2018 Request: Buy books Contact: email: hi@gmail.com Tel: 0123456789 Ordered inquiry: Order ID: 12345678 BL: 87654321 Product: 123456 Books' lines UNION ALL
  SELECT 'Date of application: 01/04/2018 Request: Retour table Contact: Rodion Raskólnikov email: hello@outlook.com Tel: 9876543210 Ordered inquiry: Order Id: 87654321 BL: 12345678 Product: 654321 Tables'
), raw_messages AS (
  SELECT lines
  FROM `project.dataset.my_table` 
  WHERE REGEXP_CONTAINS(lines, '^Date of application: ')
)
SELECT 
  REGEXP_EXTRACT(lines, r'(?i)^Date of application: ([0-9]{2}/[0-9]{2}/[0-9]{4})') AS DATE,
  REGEXP_EXTRACT(lines, r'(?i) Request: (.*?) Contact: ') AS request,
  REGEXP_EXTRACT(lines, r'(?i) Contact: (.*?) email: ') AS contact,
  REGEXP_EXTRACT(lines, r'(?i) email: (.*?) Tel: ') AS email,
  REGEXP_EXTRACT(lines, r'(?i) Tel: (.*?) Ordered inquiry: ') AS phone,
  REGEXP_EXTRACT(lines, r'(?i) Order ID: (.*?) BL: ') AS id,
  REGEXP_EXTRACT(lines, r'(?i) BL: (.*?) Product: ') AS bl,
  REGEXP_EXTRACT(lines, r'(?i) Product: (.*?)$') AS product
FROM raw_messages

结果

Row DATE        request         contact             email               phone       id          bl          product  
1   01/02/2018  Buy books       null                hi@gmail.com        0123456789  12345678    87654321    123456 Books     
2   01/04/2018  Retour table    Rodion Raskólnikov  hello@outlook.com   9876543210  87654321    12345678    654321 Tables

【讨论】：