SQL - 使用 JOIN 或 SUM 时的双倍（或三倍）计数统计信息答案

【问题标题】：SQL - Double (or Triple) counting stats when using JOIN or SUMSQL - 使用 JOIN 或 SUM 时的双倍（或三倍）计数统计信息
【发布时间】：2021-05-07 13:44:56
【问题描述】：

我对 SQL 不熟悉，目前在 StackOverflow 上阅读了很多内容。我在 BigQuery 中使用 Google Ads 数据传输。

使用以下公式将展示次数、点击次数和支出量增加一倍/三倍时遇到了问题：

SELECT
  DISTINCT campaignStatsOverall.ExternalCustomerId AS klant_id,
  AccountDescriptiveName AS klant_naam,
  campaignStatsOverall.Date AS datum,
  EXTRACT(DAY
  FROM
    campaignStatsOverall.Date) AS dag_in_de_maand,
  EXTRACT(MONTH
  FROM
    campaignStatsOverall.Date) AS maand,
  EXTRACT(QUARTER
  FROM
    campaignStatsOverall.Date) AS kwartaal,
  EXTRACT(YEAR
  FROM
    campaignStatsOverall.Date) AS jaar,
  EXTRACT(WEEK(MONDAY)
  FROM
    campaignStatsOverall.Date) AS weeknummer,
  SUM(campaignStatsOverall.Clicks) AS klikken,
  SUM(campaignStatsOverall.Impressions) AS vertoningen,
  SUM(campaignStatsOverall.Conversions) AS conversies,
  ROUND(SUM(campaignStatsOverall.Cost)/1000000,2) AS spend
FROM
  `mvhmedia.ads_mcc_mvhmedia_raw_data.p_Customer_7834578801` AS customerData
LEFT JOIN
  `mvhmedia.ads_mcc_mvhmedia_raw_data.p_CampaignBasicStats_7834578801` AS campaignStatsOverall
ON
  campaignStatsOverall.ExternalCustomerId = customerData.ExternalCustomerId
WHERE
  DATE(customerData._PARTITIONTIME) > '2020-04-30'
GROUP BY
  klant_id,
  klant_naam,
  datum
ORDER BY
  datum DESC,
  klant_naam

当我不在 SELECT 区域中使用数学时，数字是正确的。这给了我很多表中的行（由于 DISTINCT 和 SUM）：

SELECT
  DISTINCT AccountDescriptiveName AS klant_naam,
  campaignStatsOverall.ExternalCustomerId AS klant_id,
  campaignStatsOverall.Date AS datum,
  EXTRACT(DAY
  FROM
    campaignStatsOverall.Date) AS dag_in_de_maand,
  EXTRACT(MONTH
  FROM
    campaignStatsOverall.Date) AS maand,
  EXTRACT(QUARTER
  FROM
    campaignStatsOverall.Date) AS kwartaal,
  EXTRACT(YEAR
  FROM
    campaignStatsOverall.Date) AS jaar,
  EXTRACT(WEEK(MONDAY)
  FROM
    campaignStatsOverall.Date) AS weeknummer,
  campaignStatsOverall.Clicks AS klikken,
  campaignStatsOverall.Impressions AS vertoningen,
  ROUND(campaignStatsOverall.Conversions,3) AS conversies,
  ROUND(campaignStatsOverall.Cost/1000000,2) AS spend
FROM
  `mvhmedia.ads_mcc_mvhmedia_raw_data.p_Customer_7834578801` AS customerData
LEFT JOIN
  `mvhmedia.ads_mcc_mvhmedia_raw_data.p_CampaignBasicStats_7834578801` AS campaignStatsOverall
ON
  campaignStatsOverall.ExternalCustomerId = customerData.ExternalCustomerId
WHERE
  DATE(customerData._PARTITIONTIME) > '2020-04-30'
  AND campaignStatsOverall.ExternalCustomerId = customerData.ExternalCustomerId
GROUP BY
  klant_id,
  klant_naam,
  datum,
  klikken,
  vertoningen,
  conversies,
  spend
ORDER BY
  datum DESC,
  klant_naam

认为我在使用 DISTINCT、SUM 或 LEFT JOIN 时做错了。

希望有人可以帮助我。

应该是什么：

klantnaam	klikken	vertoningen	conversies	spend
client a	112	2599	7.0	33.08
client b	66	3223	6.0	26.6

它给了我什么（三倍）：

klantnaam	klikken	vertoningen	conversies	spend
client a	336	7797	21.0	99.26
client b	198	9669	18.0	79.8

似乎数据增加了三倍。

【问题讨论】：

1) 当您执行 GROUP BY 时，您很少不需要执行 SELECT DISTINCT。（当我看到这种情况时，我会扬起眉毛，想知道发生了什么。）
轻松为您提供帮助 - 简化！并提供样表数据和预期结果——minimal reproducible example。
添加结果应该是什么而不显示源数据是什么，就像要求我们执行真正的魔术一样。（你读过 Jarl 给你的链接吗？）

标签： sql google-bigquery left-join distinct

【解决方案1】：

当您使用LEFT JOIN 时，您似乎在CustomerData 和campaignStatsOverall 之间有一个1-n 连接。这意味着SUM 将对每个客户从campaignStatsOverall 链接的行求和n 倍。

为避免这种情况，我建议您查看窗口功能。使用SUM，您可以使用以下内容：

 SUM(campaignStatsOverall.Clicks) OVER (PARTITION BY customerData.ExternalCustomerId) AS klikken,

请注意，通常不能将窗口函数与 GROUP BY 子句一起使用。因此，如果您需要区分/分组来自CustomerData 的行，您应该在子查询中执行此操作（请参阅WITHclause），然后在相关子查询上应用SELECT，并使用窗口化SUM。

我敦促您始终仔细检查表连接是否存在潜在的笛卡尔效应，您应该始终依赖主键连接或明确定义的笛卡尔连接。

编辑： Google BigQuery 的窗口函数（也称为分析函数）文档：https://cloud.google.com/bigquery/docs/reference/standard-sql/analytic-function-concepts

EDIT2： 由于没有 cmets，我猜 OP 正在寻找完整的脚本。这里是：

SELECT
  DISTINCT campaignStatsOverall.ExternalCustomerId AS klant_id,
  AccountDescriptiveName AS klant_naam,
  campaignStatsOverall.Date AS datum,
  EXTRACT(DAY
  FROM
    campaignStatsOverall.Date) AS dag_in_de_maand,
  EXTRACT(MONTH
  FROM
    campaignStatsOverall.Date) AS maand,
  EXTRACT(QUARTER
  FROM
    campaignStatsOverall.Date) AS kwartaal,
  EXTRACT(YEAR
  FROM
    campaignStatsOverall.Date) AS jaar,
  EXTRACT(WEEK(MONDAY)
  FROM
    campaignStatsOverall.Date) AS weeknummer,
  SUM(campaignStatsOverall.Clicks) OVER (PARTITION BY customerData.ExternalCustomerId, customerData.AccountDescriptiveName, campaignStatsOverall.Date) AS klikken,
  SUM(campaignStatsOverall.Impressions) OVER (PARTITION BY customerData.ExternalCustomerId, customerData.AccountDescriptiveName, campaignStatsOverall.Date) AS vertoningen,
  SUM(campaignStatsOverall.Conversions) OVER (PARTITION BY customerData.ExternalCustomerId, customerData.AccountDescriptiveName, campaignStatsOverall.Date) AS conversies,
  ROUND(SUM(campaignStatsOverall.Cost) OVER (PARTITION BY customerData.ExternalCustomerId, customerData.AccountDescriptiveName, campaignStatsOverall.Date)/1000000,2) AS spend
FROM
  `mvhmedia.ads_mcc_mvhmedia_raw_data.p_Customer_7834578801` AS customerData
LEFT JOIN
  `mvhmedia.ads_mcc_mvhmedia_raw_data.p_CampaignBasicStats_7834578801` AS campaignStatsOverall
ON
  campaignStatsOverall.ExternalCustomerId = customerData.ExternalCustomerId
WHERE
  DATE(customerData._PARTITIONTIME) > '2020-04-30'
ORDER BY
  datum DESC,
  klant_naam

如您所见，使用窗口函数意味着不再有GROUP BY 子句这里。 DISTINCT 现在是有意义的，因为它的目标是区分提供了klant_id、klant_naam 和datum 的两个记录。 klikken、spend 等的分区总和只是为这三个计算，因此可以在DISTINCT 输出上区分这三个。

另一方面，如果ExternalCustomerId 是campaignStatsOverall 的键，您可以从PARTITION BY/GROUP BY 操作中删除AccountDescriptiveName。

【讨论】：