RandomizedSearchCV Pipeline 使用mutual_info_classif 选择SelectPercentile 的超参数

Question

我有什么

我有一个使用我的超参数分布运行的管道

pipe = Pipeline(steps=[
    ('scale', MinMaxScaler()),
    ('vt', VarianceThreshold()),
    ('pca', PCA(random_state=0)),
    ('select', SelectPercentile()),
    ('clf', RandomForestClassifier(random_state=0))
])

hyper_params0 = {
    'vt__threshold' : stats.distributions.uniform(0, 0.1),
    'pca__n_components' : stats.distributions.uniform(0.8, 0.19),
    'select__percentile' : stats.distributions.randint(1, 101),
    'clf__n_estimators' : stats.distributions.randint(50, 1000),
    'clf__criterion' : ['gini', 'entropy'],
    'clf__min_samples_split' : stats.distributions.uniform(0, 0.1),
    'clf__min_samples_leaf' : stats.distributions.uniform(0, 0.1),
    'clf__max_features' : ['sqrt', 'log2', None],
    'clf__bootstrap' : [True, False],
}

hyper_params=[
    {
        **hyper_params0,
        **{
            'select__score_func' : [mutual_info_classif],
        }
    },
    {
        **hyper_params0,
        **{
            'select__score_func' : [f_classif],
        }
    }
]

rscv = RandomizedSearchCV(
    estimator=pipe,
    param_distributions=hyper_params,
    n_iter=25,
    cv=5,
    scoring='f1_macro',
    n_jobs=-1,
    random_state=0,
    verbose=3
)

X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.2, random_state=0)

rscv.fit(X_train, y_train)

我想要什么

我想做的是在SelectPercentile 内的mutual_info_classif 中搜索n_neighbors 参数。

我尝试了什么

我尝试像这样编辑hyper_params：

hyper_params=[
    {
        **hyper_params0,
        **{
            'select__score_func' : [mutual_info_classif],
            'select__score_func__n_neighbors' : stats.distributions.randint(3, 15)
        }
    },
    {
        **hyper_params0,
        **{
            'select__score_func' : [f_classif],
        }
    }
]

但我收到错误 AttributeError: 'function' object has no attribute 'set_params'。我在 scikit-learn 网站 here 上遵循了一个松散的示例，但并没有走得太远。还尝试像这样使用'passthrough'：

pipe = Pipeline(steps=[
    ('scale', MinMaxScaler()),
    ('vt', VarianceThreshold()),
    ('pca', PCA(random_state=0)),
    ('select', 'passthrough'),
    ('clf', RandomForestClassifier(random_state=0))
])
...
hyper_params=[
    {
        **hyper_params0,
        **{
            'select__score_func' : [SelectPercentile(mutual_info_classif)],
            'select__score_func__n_neighbors' : stats.distributions.randint(3, 15)
        }
    },
    {
        **hyper_params0,
        **{
            'select__score_func' : [SelectPercentile(f_classif)],
        }
    }
]

但是得到错误AttributeError: 'str' object has no attribute 'set_params'。

问题

关于如何做到这一点的任何建议？

Answer 1

由于错误表明mutual_info_classif 是一个函数，因此GridSearchCV 无法使用__ 为其设置参数。 GridSearchCV 只能为支持BaseEstimator 设计的类设置参数。

首先，您需要创建一个可以将n_neighbors 作为类参数的自定义SelectPercentile。

class SelectPercentileMI(SelectPercentile):
    def __init__(self, percentile=10, n_neighbors=3):
        self.n_neighbors=n_neighbors
        super().__init__(percentile=percentile,
                         score_func=partial(mutual_info_classif, n_neighbors=3))

现在，您的问题已解决。

import numpy as np
import matplotlib.pyplot as plt
from sklearn.datasets import load_digits
from sklearn.model_selection import GridSearchCV
from sklearn.pipeline import Pipeline
from sklearn.svm import LinearSVC
from sklearn.decomposition import PCA, NMF
from sklearn.feature_selection import SelectKBest, chi2, SelectPercentile, mutual_info_classif, f_classif
from scipy import stats
from functools import partial


pipe = Pipeline([
    ('feature_selector', 'passthrough'),
    ('classify', LinearSVC(dual=False, max_iter=10000))
])

C_OPTIONS = [1, 10]
param_grid = [
    {
        'feature_selector': [SelectPercentile(f_classif)],
        'classify__C': C_OPTIONS
    },
    {
        'feature_selector': [SelectPercentileMI()],
        'feature_selector__n_neighbors' : [2,3],
        'classify__C': C_OPTIONS
    },
]

grid = GridSearchCV(pipe, n_jobs=1, param_grid=param_grid)
X, y = load_digits(return_X_y=True)
grid.fit(X, y)

grid.best_params_

{'classify__C': 10, 'feature_selector': SelectPercentileMI(n_neighbors=3, percentile=10), 'feature_selector__n_neighbors'：3}