【问题标题】:what is the purpose of FLATTEN operator in PIG LatinPIG拉丁语中FLATTEN运算符的目的是什么
【发布时间】:2015-02-13 10:31:15
【问题描述】:

A = 将“数据”加载为 (x, y);

B = 将“数据”加载为 (x, z);

C = x 同组 A,x 同组 B;

D = foreach C 生成 flatten(A), flatten(b);

E = A::x 的 D 组

上述语句中究竟做了什么,以及我们在实时场景中使用 flatten 的位置。

【问题讨论】:

标签: hadoop apache-pig


【解决方案1】:
A = load 'input1'   USING PigStorage(',') as (x, y);
(x,y) --> (1,2)(1,3)(2,3)
B = load 'input2'  USING PigStorage(',') as (x, z);`
(x,z) --> (1,4)(1,2)(3,2)*/
C = cogroup A by x, B by x;`

result:

(1,{(1,2),(1,3)},{(1,4),(1,2)})
(2,{(2,3)},{})
(3,{},{(3,2)})


D = foreach C generate group, flatten(A), flatten(B);`

when both bags flattened, the cross product of tuples are returned.  

result:
(1,1,2,1,4)
(1,1,2,1,2)
(1,1,3,1,4)
(1,1,3,1,2)  

E = group D by A::x`

here your are grouping with x column of relation A.

(1,1,2,1,4) (1,1,2,1,2) (1,1,3,1,4) (1,1,3,1,2)

【讨论】:

    猜你喜欢
    • 1970-01-01
    • 1970-01-01
    • 1970-01-01
    • 2020-07-13
    • 1970-01-01
    • 1970-01-01
    • 1970-01-01
    • 2011-01-10
    • 1970-01-01
    相关资源
    最近更新 更多