【问题标题】:Performing aggregate function on a date range在日期范围内执行聚合函数
【发布时间】:2021-09-18 17:07:46
【问题描述】:

如何在日期范围内执行聚合函数 (AVG)

源数据如下:

AccNo   Amt     Date    
1       100     2/1/2021
1       200     2/5/2021
1       300     3/3/2021
2       400     5/20/2021
2       500     5/18/2021

目标需要通过以下方法导出:

例如:要导出 avg_past_week,计算日期范围 DateDate - 7 内所有行的平均金额

同样,对于avg_past_month,它将是DateDate - 30

目标:

AccNo   Amt     Date        Avg_past_week   Avg_past_month  Avg_past_3 month
1       100     2/1/2021    100             100             100
1       200     2/5/2021    150             150             150
1       300     3/3/2021    300             250             200
2       400     5/20/2021   450             450             450
2       500     5/18/2021   500             500             500

【问题讨论】:

    标签: sql scala dataframe hive apache-spark-sql


    【解决方案1】:

    你可以使用窗口函数:

    select t.*,
           avg(amt) over (partition by accNo
                          order by datediff('2000-01-01', date)
                          range between 6 preceding and current row
                         ) as avg_7day,
           avg(amt) over (partition by accNo
                          order by datediff('2000-01-01', date)
                          range between 30 preceding and current row
                         ) as avg_30day,
           avg(amt) over (partition by accNo
                          order by datediff('2000-01-01', date)
                          range between 90 preceding and current row
                         ) as avg_91day
    from t;
    

    这会将日期转换为数字(自某个任意日期以来的天数),然后使用窗口框架作为时间段。

    请注意,回溯的天数比期间少一,因为当天也包括在内。

    【讨论】:

      【解决方案2】:
      select a.AccNo, a.Amt, a.Date,
        avg(b.Amt) as Amt7,
        avg(c.Amt) as Amt30,
        avg(d.Amt) as Amt90
       from T a 
        join T b on a.AccNo = b.AccNo and b.Date > DateAdd(day, -7, a.Date) and b.date <= a.Date
         join T c on a.AccNo = c.AccNo and c.Date > DateAdd(day, -30, a.Date) and c.date <= a.Date
         join T d on a.AccNo = d.AccNo and d.Date > DateAdd(day, -90, a.Date) and d.date <= a.Date
      group by a.AccNo, a.Amt, a.Date
      

      结果

      1   100 2021-02-01  100 100 100
      1   200 2021-02-05  150 150 150
      1   300 2021-03-03  300 250 200
      2   400 2021-05-20  450 450 450
      2   500 2021-05-18  500 500 500
      

      【讨论】:

      • 谢谢,但我的数据量很大。这会导致性能问题。有什么方法可以使用 windows 功能?
      猜你喜欢
      • 2014-01-29
      • 1970-01-01
      • 1970-01-01
      • 1970-01-01
      • 1970-01-01
      • 1970-01-01
      • 2018-09-02
      • 1970-01-01
      • 1970-01-01
      相关资源
      最近更新 更多