用于删除跨列具有相同内容的连续重复行的数据框

Question

下面的数据框，当“人”、“年”和“项目”相同时，我想删除连续重复的行。

如果原始数据框如下所示，连续时具有相同'People'、'Year'、'Project'的行将被删除。

data = {'People' : ["David","David","David","David","John","John","John"],
'Year': ["2016","2016","2017","2016","2016","2017","2017",],
'Project' : ["TN","TN","TN","TN","DJ","DM","DM"],
'Earning' : [878,682,767,620,964,610,772]}

我试过了，但它不起作用：

df_1 = df.loc[(df['People', 'Year', 'Project'].shift() != df['People', 'Year', 'Project'])]

attempt - 此行删除不连续的“David, 2016, TN, 620”

df_1 = df.drop_duplicates(subset=['People','Year','Project'])

当更改为这个时，它会保留所有行：

df_1 = df.drop_duplicates(subset=['People','Year','Project', 'Earning'])

正确的做法是什么？谢谢！

Answer 1

您可以比较DataFrame.shifted 值是否不相等，然后通过DataFrame.any 和boolean indexing 每行测试至少一个True：

cols = ['People','Year','Project']
df_1 = df[df[cols].ne(df[cols].shift()).any(axis=1)]
print (df_1)
  People  Year Project  Earning
0  David  2016      TN      878
2  David  2017      TN      767
3  David  2016      TN      620
4   John  2016      DJ      964
5   John  2017      DM      610

详情：

print (df[cols].ne(df[cols].shift()))
   People   Year  Project
0    True   True     True
1   False  False    False
2   False   True    False
3   False   True    False
4    True  False     True
5   False   True     True
6   False  False    False

print (df[cols].ne(df[cols].shift()).any(axis=1))
0     True
1    False
2     True
3     True
4     True
5     True
6    False
dtype: bool