如何找到基于权重的自定义平均值，包括处理熊猫中的nan值？答案

【问题标题】：How to find customized average which is based on weightage including handling of nan value in pandas?如何找到基于权重的自定义平均值，包括处理熊猫中的nan值？
【发布时间】：2020-11-30 08:02:17
【问题描述】：

我有一个数据框 df_ss_g 作为

ent_id,WA,WB,WC,WD
123,0.045251836,0.614582906,0.225930615,0.559766482
124,0.722324239,0.057781167,,0.123603561
125,,0.361074325,0.768542766,0.080434134
126,0.085781742,0.698045853,0.763116684,0.029084545
127,0.909758657,,0.760993759,0.998406211
128,,0.32961283,,0.90038336
129,0.714585519,,0.671905291,
130,0.151888772,0.279261613,0.641133263,0.188231227

现在我必须计算基于权重的平均值（AVG_WEIGHTAGE），即 =(WA*0.5+WB*1+WC*0.5+WD*1)/(0.5+1+0.5+1)

但是当我使用下面的方法计算它时，即

df_ss_g['AVG_WEIGHTAGE']= df_ss_g.apply(lambda x:((x['WA']*0.5)+(x['WB']*1)+(x['WC']*0.5)+(x['WD']*1))/(0.5+1+0.5+1) , axis=1)

IT 输出为，即对于 NaN 值，它将 NaN 作为 AVG_WEIGHTAGE 为 null，这是错误的。

我想要的只是分母和分子中不应考虑 null 例如

ent_id,WA,WB,WC,WD,AVG_WEIGHTAGE
128,,0.32961283,,0.90038336,0.614998095   i.e. (WB*1+WD*1)/1+1
129,0.714585519,,0.671905291,,0.693245405 i.e. (WA*0.5+WC*0.5)/0.5+0.5

【问题讨论】：

如果您使用fillna() 并将所有 NaN 填充为 0 会怎样？
@user13802115 它不起作用，因为通过使用 fillna() 它被认为是分母......这使得平均错误

标签： python pandas numpy dataframe data-science

【解决方案1】：

IIUC：

import numpy as np

weights = np.array([0.5, 1, 0.5, 1]))
values = df.drop('ent_id', axis=1)

df['AVG_WEIGHTAGE'] = np.dot(values.fillna(0).to_numpy(), weights)/np.dot(values.notna().to_numpy(), weights)


df['AVG_WEIGHTAGE']
0    0.436647
1    0.217019
2    0.330312
3    0.383860
4    0.916891
5    0.614998
6    0.693245
7    0.288001

【讨论】：

【解决方案2】：

用点积试试这个方法-

def av(t):
    #Define weights
    wt = [0.5, 1, 0.5, 1]
    
    #Create a vector with 0 for null and 1 for non null
    nulls = [int(i) for i in ~t.isna()]
    
    #Take elementwise products of the nulls vector with both weights and t.fillna(0)
    wt_new = np.dot(nulls, wt)
    t_new = np.dot(nulls, t.fillna(0))
    
    #return division
    return np.divide(t_new,wt_new)

df['WEIGHTED AVG'] = df.apply(av, axis=1)
df = df.reset_index()
print(df)

   ent_id        WA        WB        WC        WD  WEIGHTED AVG
0     123  0.045252  0.614583  0.225931  0.559766      0.481844
1     124  0.722324  0.057781       NaN  0.123604      0.361484
2     125       NaN  0.361074  0.768543  0.080434      0.484020
3     126  0.085782  0.698046  0.763117  0.029085      0.525343
4     127  0.909759       NaN  0.760994  0.998406      1.334579
5     128       NaN  0.329613       NaN  0.900383      0.614998
6     129  0.714586       NaN  0.671905       NaN      1.386491
7     130  0.151889  0.279262  0.641133  0.188231      0.420172

【讨论】：

当我实现你的逻辑时，我收到了这个错误 ValueError: invalid literal for int() with base 10: 'WA'

【解决方案3】：

归结为用0 掩盖nan 值，因此它们不会对权重或总和做出贡献：

# this is the weights
weights = np.array([0.5,1,0.5,1])

# the columns of interest
s = df.iloc[:,1:]

# where the valid values are
mask = s.notnull()

# use `fillna` and then `@` for matrix multiplication
df['AVG_WEIGHTAGE'] = (s.fillna(0) @ weights) / (mask@weights)

【讨论】：