使用范围连接减少记录数

Question

关注my question 我有以下表格，第一个（范围）包括值范围和附加列：

row  | From   |  To     | Country ....
-----|--------|---------|---------
1    | 1200   |   1500  |
2    | 2200   |   2700  |
3    | 1700   |   1900  |
4    | 2100   |   2150  |
... 

From 和 To 是 bigint 并且是独占的。 Range 表包含 180 万条记录。附加表（值）包含 270 万条记录，如下所示：

 row     | Value  | More columns....
 --------|--------|----------------
    1    | 1777   |    
    2    | 2122   |    
    3    | 1832   |    
    4    | 1340   |    
    ... 

我想创建一个如下表：

row      | Value  | From   | To    | More columns....
 --------|--------|--------|-------|---
    1    | 1777   | 1700   | 1900  |
    2    | 2122   | 2100   | 2150  |   
    3    | 1832   | 1700   | 1900  |   
    4    | 1340   | 1200   | 1500  |   
    ... 

我在以下代码中使用了左外连接：

set n=1000;

select      v.id
           ,v.val
           ,r.from_val
           ,r.to_val

from      val v
        left outer join    

 (select  r.*
                   ,floor(from_val/${hiveconf:n}) + pe.i    as match_val

            from    val_range r
                    lateral view    posexplode
                                    (
                                        split
                                        (
                                            space
                                            (
                                                cast
                                                (
                                                    floor(to_val/${hiveconf:n}) 
                                                  - floor(from_val/${hiveconf:n}) 

                                                    as int
                                                )
                                            )
                                           ,' '
                                        )
                                    ) pe as i,x
            ) r



            on      floor(v.val/${hiveconf:n})    =
                    r.match_val

where       v.val between r.from_val and r.to_val

order by    v.id       
;

但是，新表的记录数量大幅减少，大约 270 万条记录中的 31000 条记录。如果我使用left outer join 会怎样？我该如何解决？

Answer 1

假设我们有一个v.id

set n=1000;

select      v.id
           ,r.from_val
           ,r.to_val

from                    val     v 

            left join  (select      v.id
                                   ,r.from_val
                                   ,r.to_val

                        from                val     v 

                                    join    (...)   r 

                                    on      floor(v.val/${hiveconf:n})    =
                                            r.match_val

                        where       v.val between r.from_val and r.to_val
                        ) r

            on          r.id    =
                        v.id

order by    v.id       

关于 OP 请求，这里是完整的查询：

set n=1000;

select      v.id
           ,r.from_val
           ,r.to_val

from                    val     v 

            left join  (select      v.id
                                   ,r.from_val
                                   ,r.to_val

                        from                val     v 

                                    join   (select  r.*
                                                   ,floor(from_val/${hiveconf:n}) + pe.i    as match_val

                                            from    val_range r
                                                    lateral view    posexplode
                                                                    (
                                                                        split
                                                                        (
                                                                            space
                                                                            (
                                                                                cast
                                                                                (
                                                                                    floor(to_val/${hiveconf:n}) 
                                                                                  - floor(from_val/${hiveconf:n}) 

                                                                                    as int
                                                                                )
                                                                            )
                                                                           ,' '
                                                                        )
                                                                    ) pe as i,x
                                            ) r

                                    on      floor(v.val/${hiveconf:n})    =
                                            r.match_val

                        where       v.val between r.from_val and r.to_val
                        ) r

            on          r.id    =
                        v.id

order by    v.id