【发布时间】：2021-08-13 01:30:50
【问题描述】：
我是 sk-learn 管道的新手，想使用我自己的离散分箱形式。我需要根据与原始列关联的另一列的累积总和对一列值进行分类。我有一个工作函数：
def dynamic_bin(df, column, weight, minimum):
"""
Parameters
----------
df : dataframe
column : column to be binned
weight : column that will dictate the bin
minimum : minimum weight per bin
Returns
-------
df : dataframe with new binned column
"""
bins = [-np.inf]
labels = []
hold_over = []
for i in sorted(df[column].unique()):
g = df[df[column] == i].groupby(column).agg({weight:'sum'}).reset_index()
if g[weight].values[0] < minimum:
if hold_over is None:
hold_over.append(g[weight].values[0])
elif (sum(hold_over) + g[weight].values[0]) < minimum:
hold_over.append(g[weight].values[0])
elif (sum(hold_over) + g[weight].values[0]) >= minimum:
hold_over.clear()
bins.append(g[column].values[0])
labels.append(g[column].values[0])
elif g[weight].values[0] >= minimum:
bins.append(g[column].values[0])
labels.append(g[column].values[0])
bins.pop()
bins.append(np.inf)
str_column = str(column)+str("_binned")
# print(str_column)
df[str_column] = pd.cut(df[column],
bins = bins,
labels = labels)
return df
这就是我试图使它成为一个类的方式。
from sklearn.base import BaseEstimator, TransformerMixin
class dynamic_bin(BaseEstimator, TransformerMixin):
def __init__(self, weight, minimum):
self.weight = weight
self.minimum = minimum
def fit(self, X, y=None):
return self
def tranform(self, X):
"""
Parameters
----------
df : dataframe
column : column to be binned
weight : column that will dictate the bin
minimum : minimum weight per bin
Returns
-------
df : dataframe with new binned column
"""
bins = [-np.inf]
labels = []
hold_over = []
for i in sorted(df[column].unique()):
g = df[df[column] == i].groupby(column).agg({weight:'sum'}).reset_index()
if g[weight].values[0] < minimum:
if hold_over is None:
hold_over.append(g[weight].values[0])
elif (sum(hold_over) + g[weight].values[0]) < minimum:
hold_over.append(g[weight].values[0])
elif (sum(hold_over) + g[weight].values[0]) >= minimum:
hold_over.clear()
bins.append(g[column].values[0])
labels.append(g[column].values[0])
elif g[weight].values[0] >= minimum:
bins.append(g[column].values[0])
labels.append(g[column].values[0])
bins.pop()
bins.append(np.inf)
str_column = str(column)+str("_binned")
# print(str_column)
df[str_column] = pd.cut(df[column],
bins = bins,
labels = labels)
return df[str_column]
当我尝试通过以下方式实现它时，我得到了它下面的错误：
column_trans = ColumnTransformer(
[
("binned_numeric", dynamic_bin(weight = 'Exposure', minimum = 1000),
["VehAge", "DrivAge"]),
("onehot_categorical", OneHotEncoder(),
["VehBrand", "VehPower", "VehGas", "Region", "Area"]),
("passthrough_numeric", "passthrough",
["BonusMalus"]),
("log_scaled_numeric", log_scale_transformer,
["Density"]),
],
remainder="drop",
)
X = column_trans.fit_transform(df)
TypeError: All estimators should implement fit and transform, or can be 'drop' or 'passthrough' specifiers. 'dynamic_bin(minimum=1000, weight='Exposure')' (type <class 'dynamic_bin.dynamic_bin'>) doesn't.
我阅读了以下内容，但我并没有真正关注它。
Put customized functions in Sklearn pipeline
有人发现我犯的错误吗？
【问题讨论】：
-
您链接的问题是的答案。要在管道中使用函数，您需要实现
.fit()和.transform()。该问题显示了如何从 sklearn 提供的基类继承，以便为管道制作一个简单的类包装器，以利用相关函数 -
我不确定我适合这种转变。
-
看起来您正在为数据安装 bin，然后返回 bin 或 binned 数据。有关内置示例，请参阅KBinsDiscretizer
-
已更新问题以获得更多上下文。
标签： python scikit-learn pipeline