根据条件和其他列在熊猫数据框中删除重复行（汇总数据）答案

【问题标题】：Drop Duplicate rows (Summarizing the data) in pandas dataframe based on condition and other columns根据条件和其他列在熊猫数据框中删除重复行（汇总数据）
【发布时间】：2021-04-19 04:42:03
【问题描述】：

我有一个数据框df 如下：

Student_ID   Subject     Weekly_Feedback        Feedack_questions            Exam_Phase
100101       Physics       Excellent: 5         How was physics class?       Mid term
100101         NaN         NaN                  how was physics class?       Final
100101       Physics       Good: 3              How was the Presentation?    Final
100101         NaN         Excellent: 5         How Much would you rate?     Mid term
100102         NaN         NaN                  how much would you rate?     Mid term 
100102       Chemistry     Good: 3              How was chemistry class?     Final
100102         NaN         Good: 3              How was over all experience? Mid term
100102         NaN         very Good: 3         How was topics explained?    Final
100102         NaN         NaN                  How did the class go?        Mid term
100103       Mathematics   NaN                  NaN                          Final
100103       Mathematics   Good :4              NaN                          Final

我想以这样一种方式总结数据，对于每个Student_ID，必须只有一个记录（行），每个记录（行）都包含Exam_Phase 作为Final和Mid Term，如果一个Stundet_ID 只有一个Exam_Phase，那么只需要考虑那个。必须选择行以尽可能包含最少的NaN （例如：在数据框的最后4行，即：

Student_ID    Subject     Weekly_Feedback       Feedack_questions            Exam_Phase
100102       Chemistry     Good: 3              How was chemistry class?     Final
100102         NaN         Good: 3              How was over all experience? Mid term
100102         NaN         very Good: 3         How was topics explained?    Final
100102         NaN         NaN                  How did the class go?        Mid Term

对于Student_ID = 100102，此行：

100102         NaN         Good: 3              How was over all experience? Mid term

必须选择，因为当Exam_Phase = Mid term 时，该行的NaN 值较少（仅在其中一列中）。 )

输出数据框应如下所示：

Student_ID   Subject     Weekly_Feedback        Feedack_questions             Exam_Phase
100101       Physics       Excellent: 5         How was physics class?        Mid term
100101       Physics       Good: 3              How was the Presentation?     Final
100102       Chemistry     Good: 3              How was chemistry class?      Final
100102         NaN         Good: 3              How was over all experience?  Mid term
100103       Mathematics   Good :4              NaN                           Final

【问题讨论】：

标签： python python-3.x pandas dataframe nan

【解决方案1】：

这是另一种使用series.argsort 然后df.query 进行过滤和drop_duplicates 的方式

l = ['Final','Mid term']
out = (df.loc[df.isna().sum(1).argsort()].query("Exam_Phase in @l")
                      .drop_duplicates(["Student_ID",'Exam_Phase']))

print(out)

    Student_ID      Subject Weekly_Feedback             Feedack_questions  \
0       100101      Physics    Excellent: 5        How was physics class?   
2       100101      Physics         Good: 3     How was the Presentation?   
5       100102    Chemistry         Good: 3      How was chemistry class?   
6       100102          NaN         Good: 3  How was over all experience?   
10      100103  Mathematics         Good :4                           NaN   

   Exam_Phase  
0    Mid term  
2       Final  
5       Final  
6    Mid term  
10      Final

【讨论】：

【解决方案2】：

你可以使用panda.DataFrame.groupby:

(df.groupby(["Student_ID", "Exam_Phase"])
    .apply(lambda x:
        x.iloc[x.isnull().sum(axis = 1).argmin()]
    )
)

PD：为了提高可读性，我将其分多行发布，但您可以将其全部放在一行中。

【讨论】：