从 Pandas drop_duplicates() 丢弃的条目创建备份数据帧

Question

我有一个 Pandas 数据框 ds。我想从名为“名称”的特定列中删除重复的条目。

+---------+------+-------+----------+--------+
| Invoice | Name | Price |   Date   | Coupon |
+---------+------+-------+----------+--------+
|  123412 | Jim  |    50 | 12/01/17 | ALBB1  |
|  431311 | Jane |    25 | 12/02/17 | BB2    |
|  134123 | Joe  |    70 | 12/03/17 | BB2    |
|  333131 | Jim  |    85 | 12/04/17 | ALBB1  |
+---------+------+-------+----------+--------+

这是我的代码：

ds = ds.drop_duplicates(subset='Name', keep='first')

我正在使用keep='first' 选项来保留在数据框中找到的第一个实例。

我想做的是从所有丢弃的条目中创建一个单独的数据框。

所以，在这个例子中。第二个数据帧 ds2 将等于：

+---------+------+-------+----------+--------+
| Invoice | Name | Price |   Date   | Coupon |
+---------+------+-------+----------+--------+
|  333131 | Jim  |    85 | 01/04/18 | ALBB1  |
+---------+------+-------+----------+--------+

Answer 1

使用duplicated 作为布尔掩码并按boolean indexing 过滤。

注意：keep='first' 应该被省略，因为默认值

df1 = df[df.duplicated(subset='Name')]
print (df1)
   Invoice Name  Price      Date Coupon
3   333131  Jim     85  12/04/17  ALBB1

此布尔掩码可用于生成DataFrames，~ 用于反转布尔掩码：

m = df.duplicated(subset='Name')
df1 = df[m]
print (df1)
   Invoice Name  Price      Date Coupon
3   333131  Jim     85  12/04/17  ALBB1

df1 = df[~m]
print (df1)

   Invoice  Name  Price      Date Coupon
0   123412   Jim     50  12/01/17  ALBB1
1   431311  Jane     25  12/02/17    BB2
2   134123   Joe     70  12/03/17    BB2

详情：

print (m)
0    False
1    False
2    False
3     True
dtype: bool

print (~m)

0     True
1     True
2     True
3    False
dtype: bool

编辑：

也可以使用keep='last' 提取没有最后一个的所有欺骗或keep=False 提取所有欺骗值：

print (df)
   Invoice  Name  Price      Date Coupon
0   123412   Jim     50  12/01/17  ALBB1
1   431311  Jane     25  12/02/17    BB2
2   134123   Joe     70  12/03/17    BB2
3   333131   Jim     85  12/04/17  ALBB1
4   333131   Jim     86  12/04/17  ALBB2 <- added new dupe row

m = df.duplicated(subset='Name')
df11 = df[m]
print (df11)
   Invoice Name  Price      Date Coupon
3   333131  Jim     85  12/04/17  ALBB1
4   333131  Jim     86  12/04/17  ALBB2

m = df.duplicated(subset='Name', keep='last')
df12 = df[m]
print (df12)
   Invoice Name  Price      Date Coupon
0   123412  Jim     50  12/01/17  ALBB1
3   333131  Jim     85  12/04/17  ALBB1

m = df.duplicated(subset='Name', keep=False)
df13 = df[m]
print (df13)
  Invoice Name  Price      Date Coupon
0   123412  Jim     50  12/01/17  ALBB1
3   333131  Jim     85  12/04/17  ALBB1
4   333131  Jim     86  12/04/17  ALBB2