【发布时间】：2021-02-10 07:39:31
【问题描述】：
如何在 pandas 中获得真正的外连接？这意味着它实际上为您提供了整个输出，而不是组合要合并的列。在我看来，这有点愚蠢，因为这样就很难确定要连续执行哪种操作。我一直这样做是为了检测是否应该插入、更新或删除数据，但是我总是必须在列上创建额外的合并副本，这对某些数据集来说只是一堆开销（有时是大量的）。
示例：
import pandas as pd
keys = ["A","B"]
df1 = pd.DataFrame({"A":[1,2,3],"B":["one","two","three"],"C":["testThis","testThat", "testThis"],"D":[None,hash("B"),hash("C")]})
df2 = pd.DataFrame({"A":[2,3,4],"B":["two","three","four"],"C":["testThis","testThat", "testThis"], "D":[hash("G"),hash("C"),hash("D")]})
fullJoinDf = df1.merge(df2, how="outer", left_on=keys, right_on=keys, suffixes=["","_r"])
display(
fullJoinDf,
)
A B C D C_r D_r
0 1 one testThis NaN NaN NaN
1 2 two testThat -3.656526e+18 testThis -9.136326e+18
2 3 three testThis -8.571400e+18 testThat -8.571400e+18
3 4 four NaN NaN testThis -4.190116e+17
注意到它如何输出A 和B 神奇地组合到一组列。我想要的是我会在 SQL 外连接等中得到什么：
A B C D A_r B_r C_r D_r
0 1 one testThis NaN NaN NaN NaN NaN
1 2 two testThat -3.656526e+18 2 two testThis -9.136326e+18
2 3 three testThis -8.571400e+18 3 three testThat -8.571400e+18
3 NaN NaN NaN NaN 4 four testThis -4.190116e+17
为@Felipe Whitaker 编辑
使用连接：
df3 = df1.copy().set_index(keys)
df4 = df2.copy().set_index(keys)
t = pd.concat([df3,df4], axis=1)
t.reset_index(),
A B C D C D
0 1 one testThis NaN NaN NaN
1 2 two testThat -3.656526e+18 testThis -9.136326e+18
2 3 three testThis -8.571400e+18 testThat -8.571400e+18
3 4 four NaN NaN testThis -4.190116e+17
编辑示例* 鉴于答案，我将发布更多测试，因此任何偶然发现此问题的人都可以看到我在执行此操作时发现的更多“gatcha”变体。
import pandas as pd
keys = ["A","B"]
df1 = pd.DataFrame({"A":[1,2,3],"B":["one","two","three"],"C":["testThis","testThat", "testThis"],"D":[None,hash("B"),hash("C")]})
df2 = pd.DataFrame({"A":[2,3,4],"B":["two","three","four"],"C":["testThis","testThat", "testThis"], "D":[hash("G"),hash("C"),hash("D")]})
df3 = df1.copy()
df4 = df2.copy()
df3.index = df3[keys]
df4.index = df4[keys]
df5 = df1.copy().set_index(keys)
df6 = df2.copy().set_index(keys)
fullJoinDf = df5.merge(df6, how="outer", left_on=keys, right_on=keys, suffixes=["","_r"])
fullJoinDf_2 = df3.merge(df4, how="outer", left_index=True, right_index=True, suffixes=["","_r"])
t = pd.concat([df1,df2], axis=1, keys=["A","B"])
display(
df3.index,
df5.index,
fullJoinDf,
fullJoinDf_2,
t,
)
Index([(1, 'one'), (2, 'two'), (3, 'three')], dtype='object')
MultiIndex([(1, 'one'),
(2, 'two'),
(3, 'three')],
names=['A', 'B'])
A B C D C_r D_r
0 1 one testThis NaN NaN NaN
1 2 two testThat -3.656526e+18 testThis -9.136326e+18
2 3 three testThis -8.571400e+18 testThat -8.571400e+18
3 4 four NaN NaN testThis -4.190116e+17
A B C D A_r B_r C_r D_r
(1, one) 1.0 one testThis NaN NaN NaN NaN NaN
(2, two) 2.0 two testThat -3.656526e+18 2.0 two testThis -9.136326e+18
(3, three) 3.0 three testThis -8.571400e+18 3.0 three testThat -8.571400e+18
(4, four) NaN NaN NaN NaN 4.0 four testThis -4.190116e+17
A B C D A B C D
0 1 one testThis NaN 2 two testThis -9136325526401183790
1 2 two testThat -3.656526e+18 3 three testThat -8571400026927442160
2 3 three testThis -8.571400e+18 4 four testThis -419011572131270498
【问题讨论】：
-
你为什么不用
pd.concat(iter, axis = 1)？ -
@FelipeWhitaker - concat 似乎做同样的事情，请参阅编辑。
-
我真的认为它只是将它们连接起来。这个结果是反直觉的。好的，谢谢。
-
你不是要求按列连接，而不是外连接（如果至少有公共列，它只是一个连接，对吧？）？我无法理解您的示例，请编辑以澄清您的意思 “我认为这有点愚蠢，因为这样很难确定要连续执行哪种操作。”?这不是我们通常设计模式以在数据库表中具有一些主键（/id）的原因吗？如果没有，我们如何理解您的数据？
-
我看不出您的架构从具有不同不兼容 ID 的两列
"A":[1,2,3],"B":["one","two","three"]中获得了什么，如果您打算将其与连接和合并一起使用，它对我来说只是一个糟糕的架构设计。
标签： python pandas concat outer-join