使用 f1-score 对多个参数进行交叉验证

Question

我正在尝试使用 SelectKBest 进行特征选择，并使用 f1-score 进行二元分类的最佳树深度。我创建了一个记分器功能来选择最佳功能并评估网格搜索。当分类器试图适应训练数据时，会弹出“call() missing 1 required positional argument: 'y_true'”的错误。

#Define scorer
f1_scorer = make_scorer(f1_score)
#Split data into training, CV and test set
X_train, X_test, y_train, y_test = train_test_split(X,y, test_size=0.25, random_state = 0)

#initialize tree and Select K-best features for classifier   
kbest = SelectKBest(score_func=f1_scorer, k=all)
clf = DecisionTreeClassifier(random_state=0)

#create a pipeline for features to be optimized
pipeline = Pipeline([('kbest',kbest),('dt',clf)])

#initialize a grid search with features to be optimized
gs = GridSearchCV(pipeline,{'kbest__k': range(2,11), 'dt__max_depth':range(3,7)}, refit=True, cv=5, scoring = f1_scorer)

gs.fit(X_train,y_train)

#order best selected features into a single variable
selector = SelectKBest(score_func=f1_scorer, k=gs.best_params_['kbest__k'])
X_new = selector.fit_transform(X_train,y_train)  

On the fit line I get a TypeError: __call__() missing 1 required positional argument: 'y_true'.

Answer 1

问题在于您用于SelectKBest 的score_func。 score_func 是一个函数，它接受两个数组 X 和 y，并返回一对数组（分数、pvalues）或一个带有分数的数组，但在您的代码中你已经将可调用的f1_scorer 提供为score_func，它只需要你的y_true 和y_pred 并计算f1 score。您可以使用chi2、f_classif 或mutual_info_classif 之一作为您的score_func 进行分类任务。此外，SelectKBest 的参数k 中有一个小错误，它应该是"all" 而不是all。我已经修改了包含这些更改的代码，

from sklearn.tree import DecisionTreeClassifier
from sklearn.feature_selection import SelectKBest
from sklearn.pipeline import Pipeline
from sklearn.model_selection import GridSearchCV
from sklearn.feature_selection import f_classif  
from sklearn.metrics import f1_score, make_scorer
from sklearn.datasets import make_classification

X, y = make_classification(n_samples=1000, n_classes=2,
                       n_informative=4, weights=[0.7, 0.3],
                       random_state=0)

f1_scorer = make_scorer(f1_score)
#Split data into training, CV and test set
X_train, X_test, y_train, y_test = train_test_split(X,y, test_size=0.25, random_state = 0)

#initialize tree and Select K-best features for classifier   
kbest = SelectKBest(score_func=f_classif)
clf = DecisionTreeClassifier(random_state=0)

#create a pipeline for features to be optimized
pipeline = Pipeline([('kbest',kbest),('dt',clf)])
gs = GridSearchCV(pipeline,{'kbest__k': range(2,11), 'dt__max_depth':range(3,7)}, refit=True, cv=5, scoring = f1_scorer)
gs.fit(X_train,y_train)
gs.best_params_

输出

{'dt__max_depth': 6, 'kbest__k': 9}

同时修改你的最后两行如下：

selector = SelectKBest(score_func=f_classif, k=gs.best_params_['kbest__k'])
X_new = selector.fit_transform(X_train,y_train)  

希望这会有所帮助！