【发布时间】：2022-01-12 18:20:09
【问题描述】：
我正在尝试来自 page 的代码。我跑到LR (tf-idf) 的部分并得到了类似的结果
之后我决定尝试GridSearchCV。我的问题如下：
1)
#lets try gridsearchcv
#https://www.kaggle.com/enespolat/grid-search-with-logistic-regression
from sklearn.model_selection import GridSearchCV
grid={"C":np.logspace(-3,3,7), "penalty":["l2"]}# l1 lasso l2 ridge
logreg=LogisticRegression(solver = 'liblinear')
logreg_cv=GridSearchCV(logreg,grid,cv=3,scoring='f1')
logreg_cv.fit(X_train_vectors_tfidf, y_train)
print("tuned hpyerparameters :(best parameters) ",logreg_cv.best_params_)
print("best score :",logreg_cv.best_score_)
#tuned hpyerparameters :(best parameters) {'C': 10.0, 'penalty': 'l2'}
#best score : 0.7390325593588823
然后我手动计算了 f1 分数。为什么不匹配？
logreg_cv.predict_proba(X_train_vectors_tfidf)[:,1]
final_prediction=np.where(logreg_cv.predict_proba(X_train_vectors_tfidf)[:,1]>=0.5,1,0)
#https://www.statology.org/f1-score-in-python/
from sklearn.metrics import f1_score
#calculate F1 score
f1_score(y_train, final_prediction)
0.9839388145315489
- 如果我尝试
scoring='precision'为什么会出现以下错误？我不清楚主要是因为我有相对平衡的数据集 (55-45%) 和f1这需要precision正在计算没有任何问题
#lets try gridsearchcv #https://www.kaggle.com/enespolat/grid-search-with-logistic-regression
from sklearn.model_selection import GridSearchCV
grid={"C":np.logspace(-3,3,7), "penalty":["l2"]}# l1 lasso l2 ridge
logreg=LogisticRegression(solver = 'liblinear')
logreg_cv=GridSearchCV(logreg,grid,cv=3,scoring='precision')
logreg_cv.fit(X_train_vectors_tfidf, y_train)
print("tuned hpyerparameters :(best parameters) ",logreg_cv.best_params_)
print("best score :",logreg_cv.best_score_)
/usr/local/lib/python3.7/dist-packages/sklearn/metrics/_classification.py:1308: UndefinedMetricWarning: Precision is ill-defined and being set to 0.0 due to no predicted samples. Use `zero_division` parameter to control this behavior.
_warn_prf(average, modifier, msg_start, len(result))
/usr/local/lib/python3.7/dist-packages/sklearn/metrics/_classification.py:1308: UndefinedMetricWarning: Precision is ill-defined and being set to 0.0 due to no predicted samples. Use `zero_division` parameter to control this behavior.
_warn_prf(average, modifier, msg_start, len(result))
/usr/local/lib/python3.7/dist-packages/sklearn/metrics/_classification.py:1308: UndefinedMetricWarning: Precision is ill-defined and being set to 0.0 due to no predicted samples. Use `zero_division` parameter to control this behavior.
_warn_prf(average, modifier, msg_start, len(result))
/usr/local/lib/python3.7/dist-packages/sklearn/metrics/_classification.py:1308: UndefinedMetricWarning: Precision is ill-defined and being set to 0.0 due to no predicted samples. Use `zero_division` parameter to control this behavior.
_warn_prf(average, modifier, msg_start, len(result))
/usr/local/lib/python3.7/dist-packages/sklearn/metrics/_classification.py:1308: UndefinedMetricWarning: Precision is ill-defined and being set to 0.0 due to no predicted samples. Use `zero_division` parameter to control this behavior.
_warn_prf(average, modifier, msg_start, len(result))
/usr/local/lib/python3.7/dist-packages/sklearn/metrics/_classification.py:1308: UndefinedMetricWarning: Precision is ill-defined and being set to 0.0 due to no predicted samples. Use `zero_division` parameter to control this behavior.
_warn_prf(average, modifier, msg_start, len(result))
tuned hpyerparameters :(best parameters) {'C': 0.1, 'penalty': 'l2'}
best score : 0.9474200393672962
- 有没有更简单的方法来获取对火车数据的预测？我们已经有了
logreg_cv对象。我使用以下方法来恢复预测。有没有更好的方法来做同样的事情？
logreg_cv.predict_proba(X_train_vectors_tfidf)[:,1]
############################
############更新1
- 请回答上面的问题 1。在问题的评论中它说
The best score in GridSearchCV is calculated by taking the average score from cross validation for the best estimators. That is, it is calculated from data that is held out during fitting. From what I can tell, you are calculating predicted values from the training data and calculating an F1 score on that. Since the model was trained on that data, that is why the F1 score is so much larger compared to the results in the grid search
这就是我得到以下结果的原因#tuned hpyerparameters :(best parameters) {'C': 10.0, 'penalty': 'l2'} #best score : 0.7390325593588823
但是当我手动执行时，我得到
f1_score(y_train, final_prediction) 0.9839388145315489
2)
我尝试按照以下答案中的建议使用f1_micro 进行调整。没有错误信息。我仍然不清楚为什么precision 失败时f1_micro 没有失败
from sklearn.model_selection import GridSearchCV
grid={"C":np.logspace(-3,3,7), "penalty":["l2"], "solver":['liblinear','newton-cg'], 'class_weight':[{ 0:0.95, 1:0.05 }, { 0:0.55, 1:0.45 }, { 0:0.45, 1:0.55 },{ 0:0.05, 1:0.95 }]}# l1 lasso l2 ridge
#logreg=LogisticRegression(solver = 'liblinear')
logreg=LogisticRegression()
logreg_cv=GridSearchCV(logreg,grid,cv=3,scoring='f1_micro')
logreg_cv.fit(X_train_vectors_tfidf, y_train)
tuned hpyerparameters :(best parameters) {'C': 10.0, 'class_weight': {0: 0.45, 1: 0.55}, 'penalty': 'l2', 'solver': 'newton-cg'}
best score : 0.7894909688013136
【问题讨论】：
-
GridSearchCV 中的最佳分数是通过从交叉验证中获取最佳估计器的平均分数来计算的。也就是说，它是根据拟合期间保留的数据计算得出的。据我所知，您正在根据训练数据计算预测值并计算 F1 分数。由于模型是根据该数据训练的，这就是为什么 F1 分数与网格搜索结果相比要大得多的原因。
-
在 2 号上，这是一个警告，而不是错误。它告诉你y_train中有一些标签没有被预测，所以精度为0。
-
我有大约 55%-45% 的二元分类。为什么它不能预测其中一个标签？ f1 分数也可以正常工作，f1 分数需要精确
-
该模型可能无法很好地预测其中一个类。您可以使用
set(y_train) - set(final_prediction)对此进行测试。如果结果不是空集，则模型不会预测该标签。至于差异，我不确定没有看到数据，但是您可以通过在创建LogisticRegression实例时包含random_state=来使模型更具重现性。 -
因为我有 55-45% 的拆分，所以两个标签都在预测中。我之前的问题仍然存在 - f1 分数没有任何问题，f1 分数需要精度，所以精度本身应该可以工作
标签： python scikit-learn logistic-regression