如何在 GridSearchCV 中使用 Matthews 系数进行评分？

Question

我正在使用GridSearchCV 对我的机器学习结果进行超参数调整：

grid_search = GridSearchCV(estimator=xg_clf, scoring='f1', param_grid=param_grid, n_jobs=-1, cv=kfold)

但是，我的主管希望我使用 Matthews 系数进行评分，遗憾的是，这不是可用的选项之一：

>>> sorted(sklearn.metrics.SCORERS.keys())
['accuracy', 'adjusted_mutual_info_score', 'adjusted_rand_score', 'average_precision', 'balanced_accuracy', 'completeness_score', 'explained_variance', 'f1', 'f1_macro', 'f1_micro', 'f1_samples', 'f1_weighted', 'fowlkes_mallows_score', 'homogeneity_score', 'jaccard', 'jaccard_macro', 'jaccard_micro', 'jaccard_samples', 'jaccard_weighted', 'max_error', 'mutual_info_score', 'neg_brier_score', 'neg_log_loss', 'neg_mean_absolute_error', 'neg_mean_absolute_percentage_error', 'neg_mean_gamma_deviance', 'neg_mean_poisson_deviance', 'neg_mean_squared_error', 'neg_mean_squared_log_error', 'neg_median_absolute_error', 'neg_root_mean_squared_error', 'normalized_mutual_info_score', 'precision', 'precision_macro', 'precision_micro', 'precision_samples', 'precision_weighted', 'r2', 'rand_score', 'recall', 'recall_macro', 'recall_micro', 'recall_samples', 'recall_weighted', 'roc_auc', 'roc_auc_ovo', 'roc_auc_ovo_weighted', 'roc_auc_ovr', 'roc_auc_ovr_weighted', 'top_k_accuracy', 'v_measure_score']

我在文档中阅读了Demonstration of multi-metric evaluation on cross_val_score and GridSearchCV，但看起来这并不容易。

如何在使用 GridSearchCV 评分时使用 Matthews 系数？

Answer 1

关于模块 sklearn.metrics._scorer 中函数 make_scorer 的帮助：

make_scorer(score_func, *, greater_is_better=True, needs_proba=False, needs_threshold=False, **kwargs)
    Make a scorer from a performance metric or loss function.
    
    This factory function wraps scoring functions for use in
    :class:`~sklearn.model_selection.GridSearchCV` and
    :func:`~sklearn.model_selection.cross_val_score`.
    It takes a score function, such as :func:`~sklearn.metrics.accuracy_score`,
    :func:`~sklearn.metrics.mean_squared_error`,

所以解决方案是简单地使用make_scorer 并从sklearn.metrics 调用适当的函数：

grid_search = GridSearchCV(estimator=xg_clf, scoring=make_scorer(matthews_corrcoef), param_grid=param_grid, n_jobs=-1, cv=kfold)