合并熊猫数据框会重复一些数据答案

【问题标题】：Merging pandas dataframes duplicates some data合并熊猫数据框会重复一些数据
【发布时间】：2016-03-23 01:49:49
【问题描述】：

感谢您花时间阅读我的帖子。

我正在使用 Python pandas 并合并来自多个 CSV 和 TSV 文件的信息。当我执行第二次合并时，结果数据框中的数据被复制了。我假设，我错过了合并调用的一些基本内容，但我无法弄清楚。

代码：

from pandas import DataFrame, read_csv
import matplotlib.pyplot as plt
import pandas as pd 
import sys
import matplotlib

# Enable inline plotting
%matplotlib inline

# read data into dataframes
ticketdata = r'/pathto.csv'
ticketdata = r'/pathto.csv'
userdata = r'/pathto.csv'
shipmentdata = r'/pathto.tsv'

tickets_df = pd.read_csv((ticketdata), usecols=['Id',"Requester",'Created at',"Requester email",
                                                "Requester external id"])
users_df = pd.read_csv((userdata), usecols=['External ID','Printers',"Organization Title"])
shipment_df = pd.read_csv((shipmentdata), delimiter='\t', usecols=['Cust','Printer ID'])

# Clean up tickets_df & shipment_df

# Change "Requester external id" to "External ID" to support the merge
tickets_df.columns = ['Ticket Id',"Requester","External ID","Requester email",'Created at']
shipment_df.columns = ['VAR','Printers']
# Change column order for the sake of readability
tickets_df = tickets_df[['Ticket Id','Requester','Created at',"Requester email","External ID"]]

# Replace NaN in External ID with 0 and merge data
tickets_df.fillna(0, inplace=True)
merge1_df = pd.merge(tickets_df, users_df, on=['External ID'], how='left')
merge1_df = merge1_df[['Ticket Id','Created at',"Organization Title",'Requester',"Requester email","External ID",'Printers']]
merge2_df = pd.merge(merge1_df, shipment_df, on=['Printers'], how='left')

merge1_df 看起来符合预期（某些值应为 NaN）：

    Ticket Id   Created at  Organization Title  Requester   Requester email     External ID     Printers
0   1   2014-08-21 18:19    NaN     dude    dude@dude.com   0   NaN
1   2   2014-09-09 12:04    NaN     dude1   duke1@dude.com  0   NaN
2   3   2014-09-09 12:04    NaN     dude2   duke2@dude.com  0   NaN
3   4   2014-09-09 12:04    NaN     dude3   duke3@dude.com  0   NaN

merge2_df 包含数千个骗子：

    Ticket Id   Created at  Organization Title  Requester   Requester email     External ID     Printers
0   1   2014-08-21 18:19    NaN     dude    dude@dude.com   0   NaN
1   1   2014-08-21 18:19    NaN     dude    dude@dude.com   0   NaN
2   1   2014-08-21 18:19    NaN     dude    dude@dude.com   0   NaN
3   1   2014-08-21 18:19    NaN     dude    dude@dude.com   0   NaN

知道我是如何搞砸 merge2_df 的吗？

【问题讨论】：

代码过多，无法尝试通读，我们无权访问您的 CSV。但大概如果您在合并后有重复项，那么您在合并前就有重复键。

标签： python pandas merge

【解决方案1】：

问题在于 shipping_df 数据框中的 NaN 值。我添加了以下内容以将 NaN 替换为 0，并解决了 merge2_df 中的重复条目

shipment_df.fillna(0, inplace=True)

【讨论】：