如何在管道中使用适当的 FunctionTransformer 制作 GridSearchCV？

Question

我正在尝试使用 GridSearchCV 创建一个管道来过滤数据（使用 iforest）并使用 StandarSclaler+MLPRegressor 执行回归。

我制作了一个 FunctionTransformer 以将我的 iForest 过滤器包含在管道中。我还为 iForest 过滤器定义了一个参数网格（使用 kw_args 方法）。

看起来一切正常，但是当不合适时，什么也没有发生……没有错误消息。什么都没有。

之后，当我想进行预测时，我收到消息：“尚未安装此 RandomizedSearchCV 实例”

from sklearn.preprocessing import FunctionTransformer

#Definition of the function auto_filter using the iForest algo
def auto_filter(DF, conta=0.1):
    #iForest made on the DF dataframe
    iforest = IsolationForest(behaviour='new', n_estimators=300, max_samples='auto', contamination=conta)
    iforest = iforest.fit(DF)

    # The DF (dataframe in input) is filtered taking into account only the inlier observations

data_filtered = DF[iforest.predict(DF) == 1]

    # Only few variables are kept for the next step (regression by MLPRegressor)
    # this function delivers X_filtered and y
    X_filtered = data_filtered[['SessionTotalTime','AverageHR','MaxHR','MinHR','EETotal','EECH','EEFat','TRIMP','BeatByBeatRMSSD','BeatByBeatSD','HFAverage','LFAverage','LFHFRatio','Weight']]
    y = data_filtered['MaxVO2']
    return (X_filtered, y)

#Pipeline definition ('auto_filter' --> 'scaler' --> 'MLPRegressor')    
pipeline_steps = [('auto_filter', FunctionTransformer(auto_filter)), ('scaler', StandardScaler()), ('MLPR', MLPRegressor(solver='lbfgs', activation='relu', early_stopping=True, n_iter_no_change=20, validation_fraction=0.2, max_iter=10000))]

#Gridsearch Definition with differents values of 'conta' for the first stage of the pipeline ('auto_filter)
parameters = {'auto_filter__kw_args': [{'conta': 0.1}, {'conta': 0.2}, {'conta': 0.3}], 'MLPR__hidden_layer_sizes':[(sp_randint.rvs(1, nb_features, 1),), (sp_randint.rvs(1, nb_features, 1), sp_randint.rvs(1, nb_features, 1))], 'MLPR__alpha':sp_rand.rvs(0, 1, 1)}   

pipeline = Pipeline(pipeline_steps)

estimator = RandomizedSearchCV(pipeline, parameters, cv=5, n_iter=10)
estimator.fit(X_train, y_train)

Answer 1

您可以尝试手动一步步运行来查找问题：

auto_filter_transformer = FunctionTransformer(auto_filter)
X_train = auto_filter_transformer.fit_transform(X_train)

scaler = StandardScaler()
X_train = scaler.fit_transform(X_train)

MLPR = MLPRegressor(solver='lbfgs', activation='relu', early_stopping=True, n_iter_no_change=20, validation_fraction=0.2, max_iter=10000)
MLPR.fit(X_train, y_train)

如果每个步骤都运行良好，请构建管道。检查管道。如果运行良好，请尝试使用RandomizedSearchCV。

Answer 2

FunctionTransformer 的 func 参数应该是一个接受 与transform 方法相同的参数（类似于数组的X 的形状 (n_samples, n_features) 和 kwargs for func) 并返回转换后的 X 相同的形状。您的函数auto_filter 不符合这些要求。

此外，scikit-learn 的异常/异常值检测技术不能 用作 scikit-learn 管道中的中间步骤，因为管道组装 一个或多个转换器和一个可选的最终估计器。 IsolationForest 或， 比如说，OneClassSVM 不是转换器：它实现了fit 和predict。 因此，一个可能的解决方案是分别切断可能的异常值并构建 由转换器和回归器组成的管道：

>>> import warnings
>>> from sklearn.exceptions import ConvergenceWarning
>>> warnings.filterwarnings(category=ConvergenceWarning, action='ignore')
>>> import numpy as np
>>> from scipy import stats
>>> from sklearn.datasets import make_regression
>>> from sklearn.ensemble import IsolationForest
>>> from sklearn.model_selection import RandomizedSearchCV
>>> from sklearn.neural_network import MLPRegressor
>>> from sklearn.pipeline import Pipeline
>>> from sklearn.preprocessing import StandardScaler
>>> X, y = make_regression(n_samples=50, n_features=2, n_informative=2)
>>> detect = IsolationForest(contamination=0.1, behaviour='new')
>>> inliers_mask = detect.fit_predict(X) == 1
>>> pipe = Pipeline([('scale', StandardScaler()),
...                  ('estimate', MLPRegressor(max_iter=500, tol=1e-5))])
>>> param_distributions = dict(estimate__alpha=stats.uniform(0, 0.1))
>>> search = RandomizedSearchCV(pipe, param_distributions,
...                             n_iter=2, cv=3, iid=True)
>>> search = search.fit(X[inliers_mask], y[inliers_mask])

问题是您将无法优化 IsolationForest。处理它的一种方法是定义超参数空间 对于森林，使用ParameterSampler 采样超参数或 ParameterGrid，预测内点并拟合随机搜索：

>>> from sklearn.model_selection import ParameterGrid
>>> forest_param_dict = dict(contamination=[0.1, 0.15, 0.2])
>>> forest_param_grid = ParameterGrid(forest_param_dict)
>>> for sample in forest_param_grid:
...     detect = detect.set_params(contamination=sample['contamination'])
...     inliers_mask = detect.fit_predict(X) == 1
...     search.fit(X[inliers_mask], y[inliers_mask])