TypeError：如果没有指定评分，则传递的估计器应该有一个'score'方法答案

【问题标题】：TypeError: If no scoring is specified, the estimator passed should have a 'score' methodTypeError：如果没有指定评分，则传递的估计器应该有一个'score'方法
【发布时间】：2020-12-26 12:31:36
【问题描述】：

我使用 PyTorch 神经网络已经有一段时间了。我决定要添加一个排列特征重要性评分器，这开始导致一些问题。

我得到“TypeError：如果没有指定评分，则传递的估计器应该有一个'score'方法。估计器[未初始化]（ module=main.run..MultiLayerPredictor'>, ) 没有。" - 错误消息。这是我的代码：

class MultiLayerPredictor(torch.nn.Module):
    def __init__(self, input_shape=9152, output_shape=1, hidden_dim=1024, **kwargs):
        super().__init__()
        self.fc1 = torch.nn.Linear(in_features=input_shape, out_features=hidden_dim)
        self.fc2 = torch.nn.Linear(in_features=hidden_dim, out_features=hidden_dim)
        self.fc3 = torch.nn.Linear(in_features=hidden_dim, out_features=output_shape)

    def forward(self, x):
        l1 = torch.relu(self.fc1(x))
        l2 = torch.relu(self.fc2(l1))
        return torch.sigmoid(self.fc3(l2)).reshape(-1)

print("Moving to wrapping the neural net")
net = NeuralNet(
    MultiLayerPredictor,
    criterion=nn.MSELoss,
    max_epochs=10,
    optimizer=optim.Adam,
    lr=0.1,
    iterator_train__shuffle=True
)

print("Moving to finding optimal hyperparameters")

lr = (10**np.random.uniform(-5,-2.5,1000)).tolist()
params = {
    'optimizer__lr': lr,
    'max_epochs':[300,400,500],
    'module__num_units': [14,20,28,36,42],
    'module__drop' : [0,.1,.2,.3,.4]
}

gs = RandomizedSearchCV(net,params,refit=True,cv=3,scoring='neg_mean_squared_error',n_iter=100)
gs.fit(X_train_scaled,y_train);

def report(results, n_top=3):
    for i in range(1, n_top + 1):
        candidates = np.flatnonzero(results['rank_test_score'] == i)
    for candidate in candidates:
        print("Model with rank: {0}".format(i))
        print("Mean validation score: {0:.3f} (std: {1:.3f})".format(
              results['mean_test_score'][candidate],
              results['std_test_score'][candidate]))
        print("Parameters: {0}".format(results['params'][candidate]))
        print("")

print(report(gs.cv_results_,10))

epochs = [i for i in range(len(gs.best_estimator_.history))]
train_loss = gs.best_estimator_.history[:,'train_loss']
valid_loss = gs.best_estimator_.history[:,'valid_loss']

plt.plot(epochs,train_loss,'g-');
plt.plot(epochs,valid_loss,'r-');
plt.title('Training Loss Curves');
plt.xlabel('Epochs');
plt.ylabel('Mean Squared Error');
plt.legend(['Train','Validation']);
plt.show()

r = permutation_importance(net, X_test, y_test, n_repeats=30,random_state=0)

for i in r.importances_mean.argsort()[::-1]:
    if r.importances_mean[i] - 2 * r.importances_std[i] > 0:
        print(f"{metabolites.feature_names[i]:<8}"
              f"{r.importances_mean[i]:.3f}"
              f" +/- {r.importances_std[i]:.3f}")

y_pred_acc = gs.predict(X_test)
print('Accuracy : ' + str(accuracy_score(y_test,y_pred_acc)))

Stacktrace 会指出错误源于我设置排列重要性的行。我该如何解决这个问题？

完整的堆栈跟踪：

*Traceback (most recent call last):
  File "//ad..fi/home/h//Desktop/neuralnet/neuralnet_wrapped.py", line 141, in <module>
    run()
  File "//ad..fi/home/h//Desktop/neuralnet/neuralnet_wrapped.py", line 119, in run
    r = permutation_importance(net, X_test, y_test,
  File "C:\Users\\AppData\Roaming\Python\Python38\site-packages\sklearn\utils\validation.py", line 73, in inner_f
    return f(**kwargs)
  File "C:\Users\\AppData\Roaming\Python\Python38\site-packages\sklearn\inspection\_permutation_importance.py", line 132, in permutation_importance
    scorer = check_scoring(estimator, scoring=scoring)
  File "C:\Users\\AppData\Roaming\Python\Python38\site-packages\sklearn\utils\validation.py", line 73, in inner_f
    return f(**kwargs)
  File "C:\Users\\AppData\Roaming\Python\Python38\site-packages\sklearn\metrics\_scorer.py", line 425, in check_scoring
    raise TypeError(
TypeError: If no scoring is specified, the estimator passed should have a 'score' method. The estimator <class 'skorch.net.NeuralNet'>[uninitialized](
  module=<class '__main__.run.<locals>.MultiLayerPredictor'>,
) does not.*

【问题讨论】：

能否请您发布完整的堆栈跟踪？
@AaronKeesing 我将堆栈跟踪添加到原始帖子中，因为它很长
您好，请提供您的代码的最小可重现示例来重现您的问题。目前，例如数据“X_train_scaled”没有定义。看看这个：stackoverflow.com/help/minimal-reproducible-example

标签： python scikit-learn pytorch skorch

【解决方案1】：

来自docs：

NeuralNet 仍然没有评分方法。如果你需要它，你必须自己实现它。

这就是问题所在。正如错误所说，NeuralNet 没有 score 方法。并且文档说“你必须自己实现它”。您也可以查看source-code。

【讨论】：

【解决方案2】：

正如Berriel 所说，这会失败，因为您的神经网络实例没有实现score() 方法。这是默认值，因为不清楚应该为任意学习任务返回什么分数。

这也发生在 sklearn 网格搜索中，您通过传递 scoring='neg_mean_squared_error' 绕过了这一点。您也可以在这里执行此操作：

r = permutation_importance(net, X_test, y_test, 
        scoring='neg_mean_squared_error', n_repeats=30, random_state=0)

或者说因为你也需要在其他地方评分，你可以自己实现score方法：

class MyNet(NeuralNetwork):
    def score(self, X, y):
        y = self.predict(X)
        return sklearn.metrics.mean_squared_error(y, y_pred)

【讨论】：