如何忽略 Hive Lead 函数中的空值

Question

我在查询中有以下语句在 Presto 中运行良好：

CASE  
    WHEN  cte_v1.id_agent is not null 
         THEN cte_v1.id_agent
    WHEN  cte_v1.id_agent is null
          and lead(cte_v1.interaction_channel,1) ignore nulls over(partition by id_ticket, interaction_channel order by cte_v1.interaction_start_time) = 'messaging' 
        THEN lead(cte_v1.id_agent,1) ignore nulls over(partition by id_ticket, interaction_channel order by cte_v1.interaction_start_time) 
END as agent

但是，当我尝试在 Hive 中操作此查询时，我收到以下错误 '失败：ParseException 第 12:39 行在语句中的 'nulls' 附近的 'ignore' 处缺少 KW_THEN'

如前所述，此查询在 Presto 中运行良好，并通过重新创建 id_agent 列并用下一个非空值替换任何空代理 ID 来提供正确的结果

agent	id_agent	interaction_start_time	interaction_end_time	inbound_or_outbound	interaction_channel
364920063	364920063	2020-09-16 6:38:47	2020-09-16 6:38:47	Outbound	messaging
364920063	NULL	2020-09-16 12:37:54	2020-09-16 12:37:54	Inbound	messaging
364920063	364920063	2020-09-17 8:02:07	2020-09-17 8:02:07	Outbound	messaging
191918955	191918955	2020-09-17 8:16:36	2020-09-17 8:25:45	Outbound	messaging
191918955	NULL	2020-09-17 11:16:47	2020-09-17 11:16:47	Inbound	messaging
191918955	NULL	2020-09-17 16:02:35	2020-09-17 16:11:30	Inbound	messaging
191918955	191918955	2020-09-20 2:13:14	2020-09-20 2:13:14	Outbound	messaging
191918955	191918955	2020-09-20 2:44:40	2020-09-20 2:46:09	Inbound	messaging
191918955	191918955	2020-09-20 7:13:40	2020-09-20 7:13:40	Outbound	messaging

Answer 1

目前有两个关于将 IGNORE NULLs/RESPECT NULLs 添加到 LEAD/LAG 的开放 JIRA：HIVE-18145 和 HIVE-24869

但您的问题似乎可以使用first_value 轻松解决：

如果要查找下一个非空值，可以使用 first_value 函数和第二个可选参数 TRUE，这意味着忽略 NULL。

示例（使用部分数据示例，时间戳固定为基于 24 小时可排序），您可以添加 partition by 和条件检查等来采用它，此演示是展示其工作原理的最小示例：

with mytable as
(
select 364920063  id_agent ,'2020-09-16 06:38:47' interaction_start_time union all
select NULL        ,'2020-09-16 12:37:54' union all
select 364920063   ,'2020-09-17 08:02:07' union all
select 191918955   ,'2020-09-17 08:16:36' union all
select NULL        ,'2020-09-17 11:16:47' union all
select NULL        ,'2020-09-17 16:02:35' union all
select 191918955   ,'2020-09-20 02:13:14'
)

select id_agent, interaction_start_time,
       --get first not null value in the frame between current row and unbounded following 
       first_value(id_agent,true) over(order by interaction_start_time rows between current row and unbounded following) agent
from mytable

结果：

id_agent    interaction_start_time  agent
364920063   2020-09-16 06:38:47     364920063
NULL        2020-09-16 12:37:54     364920063
364920063   2020-09-17 08:02:07     364920063
191918955   2020-09-17 08:16:36     191918955
NULL        2020-09-17 11:16:47     191918955
NULL        2020-09-17 16:02:35     191918955
191918955   2020-09-20 02:13:14     191918955