不使用 HLL 或 UDF 跨 Bigquery 数组的简单不同计数

Question

与此处的示例一样，我想在 BigQuery 数组中区分计数：Distinct Count across Bigquery arrays

但是，我有一些额外的要求，使该帖子中提供的解决方案对我来说是可行的：

• 解决方案必须不使用 UDF（太慢）
• 解决方案必须不使用 HLL 函数（必须准确）
• 解决方案必须不使用linked 解决方案上显示的SELECT from SELECT 模式，因为它需要聚合最终用户使用的灵活维度组BI 工具

所以，虽然这个扩展示例（包含用户作为分组维度）使用 HLL：

#standardSQL
WITH
  test AS (
  SELECT
    'A' AS User, DATE('2018-01-01') AS ReportDate, 2 AS value, [1,2,3] AS key
  UNION ALL
  SELECT
    'A' AS User, DATE('2018-01-02') AS ReportDate, 3 AS value, [1,4,5] AS key
  UNION ALL
  SELECT
    'B' AS User, DATE('2018-01-02') AS ReportDate, 4 AS value, [4,5,6,7,8] AS key
  UNION ALL
  SELECT
    'B' AS User, DATE('2018-01-02') AS ReportDate, 5 AS value, [3,4,5,6,7] AS key )
SELECT
  User,
  SUM(value) total_value,
  HLL_COUNT.MERGE((
    SELECT
      HLL_COUNT.INIT(key)
    FROM
      UNNEST(key) key)) AS unique_key_count
FROM
  test
GROUP BY
  user

我需要一个能够满足上述要求的不同聚合数组计数的版本。

再次，这意味着如果我仅在 ReportDate、User / ReportDate 的组合或此示例扩展了其他维度的场景上进行分组，它也应该可以正常工作.

Answer 1

#standardSQL
WITH test AS
(
  SELECT 'A' AS User, DATE('2018-01-01') AS ReportDate, 2 AS value, [1,2,3] AS key UNION ALL
  SELECT 'A' AS User, DATE('2018-01-02') AS ReportDate, 3 AS value, [1,4,5] AS key UNION ALL
  SELECT 'B' AS User, DATE('2018-01-02') AS ReportDate, 4 AS value, [4,5,6,7,8] AS key UNION ALL
  SELECT 'B' AS User, DATE('2018-01-02') AS ReportDate, 5 AS value, [3,4,5,6,7] AS key  
)
SELECT 
  User,
  SUM(IF(flag=0, value, 0)) total_value,
  COUNT(DISTINCT key) unique_key_count
FROM test, UNNEST(key) key WITH OFFSET flag
GROUP BY User   

结果

Row User    total_value unique_key_count     
1   A       5           5    
2   B       9           6