超参数调优后精度保持不变

Question

我试图对参数进行超调，但是在我这样做之后，准确度分数根本没有改变，我做错了什么？

 # Log reg
from sklearn.linear_model import LogisticRegression
logreg = LogisticRegression(C=0.3326530612244898,max_iter=100,tol=0.01)
logreg.fit(X_train,y_train)

from sklearn.metrics import confusion_matrix
​
y_pred = logreg.predict(X_test)
​
print('Accuracy of log reg is: ', logreg.score(X_test,y_test))
​
confusion_matrix(y_test,y_pred)
 # 0.9181286549707602 - acurracy before tunning

输出：

Accuracy of log reg is:  0.9181286549707602
array([[ 54,   9],
       [  5, 103]])

这是我使用 Grid Search CV：

from sklearn.model_selection import GridSearchCV
params ={'tol':[0.01,0.001,0.0001],
        'max_iter':[100,150,200],
        'C':np.linspace(1,20)/10}

grid_model = GridSearchCV(logreg,param_grid=params,cv=5)
grid_model_result = grid_model.fit(X_train,y_train)
print(grid_model_result.best_score_,grid_model_result.best_params_)

输出：

0.8867405063291139 {'C': 0.3326530612244898, 'max_iter': 100, 'tol': 0.01}

Answer 1

问题在于，在第一个块中，您评估模型在测试集上的性能，而在 GridSearchCV 中，您只查看超参数优化后在训练集上的性能。

下面的代码显示，当用于预测测试集标签时，这两个过程在准确度方面表现同样出色（~0.93）。

注意，您可能需要考虑将超参数网格与其他求解器和更大范围的max_iter 一起使用，因为我收到了收敛警告。

# Load packages
import numpy as np
import pandas as pd 
from sklearn.model_selection import train_test_split
from sklearn.linear_model import LogisticRegression
from sklearn.model_selection import GridSearchCV
from sklearn import metrics

# Load the dataset and split in X and y
df = pd.read_csv('Breast_cancer_data.csv')
X = df.iloc[:, 0:5]
y = df.iloc[:, 5]

# Perform train and test split (80/20)
X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.2, random_state=42)

# Initialize a model
Log = LogisticRegression(n_jobs=-1)

# Initialize a parameter grid
params = [{'tol':[0.01,0.001,0.0001],
        'max_iter':[100,150,200],
        'C':np.linspace(1,20)/10}]

# Perform GridSearchCV and store the best parameters
grid_model = GridSearchCV(Log,param_grid=params,cv=5)
grid_model_result = grid_model.fit(X_train,y_train)
best_param = grid_model_result.best_params_

# This step is only to prove that both procedures actually result in the same accuracy score
Log2 = LogisticRegression(C=best_param['C'], max_iter=best_param['max_iter'], tol=best_param['tol'], n_jobs=-1)
Log2.fit(X_train, y_train)

# Perform two predictions one straight from the GridSearch and the other one with manually inputting the best params
y_pred1 = grid_model_result.best_estimator_.predict(X_test)
y_pred2 = Log2.predict(X_test)

# Compare the accuracy scores and see that both are the same
print("Accuracy:",metrics.accuracy_score(y_test, y_pred1))
print("Accuracy:",metrics.accuracy_score(y_test, y_pred2))