如何以有效和动态的方式比较数据框中两组列的差异

Question

我有两个数据框 df1 和 df2 具有相同的列集。我将 2 个 dfs 合并在一起并想要

1. 计算 score1 和 score2 的分数差异
2. 查看区域列是否相同。

希望的结果是 df_final 中包含“score1_diff”、“socre2_diff”和“regional_diff”（如附图所示）。我使用下面显示的代码在 df_final 中创建了 'score1_diff'、'score2_diff' 和 'regional_diff' 列。

但是，在我读取的数据框中，我有超过 30 多个分数列和 10 多个区域列，不时会有更多的分数列和区域列添加到数据框中。除了一一创建这些列之外，还有什么是实现相同结果的有效或动态方法？

import pandas as pd 
pd.set_option('display.max_columns', None)

df1 = { 'Name':['George','Andrea','micheal','Ann',
            'maggie','Ravi','Xien','Jalpa'],
   'region':['a','a','a','a','b','b','b','b'],
    'score1':[63,42,55,70,38,77,86,99],
    'score2':[45,74,44,89,69,49,72,98]}

df2 = { 'Name':['George','Andrea','micheal', 'Matt',
            'maggie','Ravi','Xien','Jalpa'],
   'region':['a','b','a','a','a','b','b','a'],
    'score1':[62,47,55,74,32,77,86,77],
    'score2':[45,78,44,89,66,49,72,73]}

df1=pd.DataFrame(df1)

df2=pd.DataFrame(df2)

df_all = pd.merge(df1,df2,how='outer',indicator=True, on='Name',suffixes=('_df1','_df2'))

df_final=df_all.copy()
df_final['score1_diff']=df_final['score1_df1']-df_final['score1_df2']
df_final['score2_diff']=df_final['score2_df1']-df_final['score2_df2']
df_final['regional_diff']=df_final['region_df1']==df_final['region_df2']

谢谢

Answer 1

从df_final=df_all.copy()开始

df_final=df_all.copy()

#You can filter columns by the prefix; and remove the region column. Next, find the number of columns and rename them because they have to be the same name.

num_columns = len(df_final.filter(regex='_df1').iloc[:, 1:].columns)
cols = ['score{}_diff'.format(i) for i in range(1, 1 + num_columns)]

#get df1 and df2 -- NOTE: the order of the columns must be the same or you'll need to do that first. don't want score1 - score2, for instance. This assumes all columns are aligned correctly.

dfa = df_final.filter(regex='_df1').iloc[:, 1:]
dfa.columns=cols
dfb = df_final.filter(regex='_df2').iloc[:, 1:]
dfb.columns=cols

df_diff = dfa - dfb

df_final = pd.concat([df_final, df_diff], axis=1)
df_final['regional_diff']=df_final['region_df1']==df_final['region_df2']
df_final

      Name region_df1  score1_df1  score2_df1 region_df2  score1_df2  \
0   George          a      63.000      45.000          a      62.000
1   Andrea          a      42.000      74.000          b      47.000
2  micheal          a      55.000      44.000          a      55.000
3      Ann          a      70.000      89.000        NaN         NaN
4   maggie          b      38.000      69.000          a      32.000
5     Ravi          b      77.000      49.000          b      77.000
6     Xien          b      86.000      72.000          b      86.000
7    Jalpa          b      99.000      98.000          a      77.000
8     Matt        NaN         NaN         NaN          a      74.000

   score2_df2      _merge  score1_diff  score2_diff  regional_diff
0      45.000        both        1.000        0.000           True
1      78.000        both       -5.000       -4.000          False
2      44.000        both        0.000        0.000           True
3         NaN   left_only          NaN          NaN          False
4      66.000        both        6.000        3.000          False
5      49.000        both        0.000        0.000           True
6      72.000        both        0.000        0.000           True
7      73.000        both       22.000       25.000          False
8      89.000  right_only          NaN          NaN          False

Answer 2

根据 Jonathan 的想法，以下是适合我的解决方案。一般的想法是创建2个df，所有列的后缀分别为_df1和_df2，然后用_diff替换后缀，以便从另一个中减去一个df。

这种方法与 Jonathan 的解决方案相同，只是我使用特定命名约定过滤所有列，而不是使用 iloc 索引。

df_final=df_all.copy()

num_current= [itm + '_df1' for itm in num_col]
num_previous =[itm + '_df2' for itm in num_col]


dfx = df_final[num_current]
dfx.columns = dfx.columns.str.replace('_df1', '_diff')

dfy = df_final[num_previous]
dfy.columns = dfy.columns.str.replace('_df2', '_diff')

df_diff = dfx - dfy

df_all = pd.concat([df_final, df_diff], axis=1)