如何解决此 sklearn 代码中的“ValueError: not enough values to unpack”？答案

【问题标题】：How do I resolve the "ValueError: not enough values to unpack" in this sklearn code?如何解决此 sklearn 代码中的“ValueError: not enough values to unpack”？
【发布时间】：2020-05-30 19:55:28
【问题描述】：

为 Titanic Kaggle 数据设置基本管道，但出现错误：ValueError: not enough values to unpack (expected 3, got 2)。

虽然有一些关于此错误的 SO 帖子，但它们并没有完全一致，也没有提供可靠的方法来诊断这个问题，更不用说解决方案了。不幸的是，错误的回溯是模糊的。我已经注释掉了大部分行并重新运行代码以查看是否可以缩小范围

以下是数据示例：

PassengerId Pclass  Sex     Age     SibSp   Parch   Fare    Embarked    Title
892           3     male    34.5      0       0     7.8292      Q       Mr
893           3     female  47.0      1       0     7.0000      S       Mrs
894           2     male    62.0      0       0     9.6875      Q       Mr
895           3     male    27.0      0       0     8.6625      S       Mr
896           3     female  22.0      1       1     12.2875     S       Mrs

这是我正在使用的管道代码：

from sklearn.pipeline import Pipeline

from sklearn.compose import ColumnTransformer
from sklearn.impute import SimpleImputer
from sklearn.preprocessing import OneHotEncoder

from sklearn.linear_model import LogisticRegression
from sklearn.model_selection import GridSearchCV


#Create a column transformer to impute missing values and one-hot encode specific columns
col_transform = ColumnTransformer(transformers=[
                                                ('one_hot', OneHotEncoder(categories='auto', sparse=False), ['Sex', 'Embarked', 'Title']), 
                                                ('imputer', SimpleImputer(strategy='median'))], 
                                  remainder='passthrough')

#Create a pipeline for the column transformer and logistic regressor
pipe = Pipeline(steps=[('encoding', col_transform),
                       ('logistic_reg', LogisticRegression(max_iter=10000, tol=0.1))])
param_grid = {
#     'logistic_reg__class_weight':[None, 'balanced'],
#     'logistic_reg__solver':['lbfgs', 'liblinear', 'newton-cg']
}

clf = GridSearchCV(pipe, param_grid=param_grid, cv=5, verbose=True, n_jobs=-1)

best_clf=clf.fit(X_train, y_train)

问题：任何想法为什么我会收到上述错误？

编辑：这是完整的错误

_RemoteTraceback                          Traceback (most recent call last)
_RemoteTraceback: 
"""
Traceback (most recent call last):
  File "C:\Users\Lofgran\AppData\Local\Continuum\anaconda3\lib\site-packages\joblib\externals\loky\process_executor.py", line 418, in _process_worker
    r = call_item()
  File "C:\Users\Lofgran\AppData\Local\Continuum\anaconda3\lib\site-packages\joblib\externals\loky\process_executor.py", line 272, in __call__
    return self.fn(*self.args, **self.kwargs)
  File "C:\Users\Lofgran\AppData\Local\Continuum\anaconda3\lib\site-packages\joblib\_parallel_backends.py", line 567, in __call__
    return self.func(*args, **kwargs)
  File "C:\Users\Lofgran\AppData\Local\Continuum\anaconda3\lib\site-packages\joblib\parallel.py", line 225, in __call__
    for func, args, kwargs in self.items]
  File "C:\Users\Lofgran\AppData\Local\Continuum\anaconda3\lib\site-packages\joblib\parallel.py", line 225, in <listcomp>
    for func, args, kwargs in self.items]
  File "C:\Users\Lofgran\AppData\Local\Continuum\anaconda3\lib\site-packages\sklearn\model_selection\_validation.py", line 516, in _fit_and_score
    estimator.fit(X_train, y_train, **fit_params)
  File "C:\Users\Lofgran\AppData\Local\Continuum\anaconda3\lib\site-packages\sklearn\pipeline.py", line 352, in fit
    Xt, fit_params = self._fit(X, y, **fit_params)
  File "C:\Users\Lofgran\AppData\Local\Continuum\anaconda3\lib\site-packages\sklearn\pipeline.py", line 317, in _fit
    **fit_params_steps[name])
  File "C:\Users\Lofgran\AppData\Local\Continuum\anaconda3\lib\site-packages\joblib\memory.py", line 355, in __call__
    return self.func(*args, **kwargs)
  File "C:\Users\Lofgran\AppData\Local\Continuum\anaconda3\lib\site-packages\sklearn\pipeline.py", line 716, in _fit_transform_one
    res = transformer.fit_transform(X, y, **fit_params)
  File "C:\Users\Lofgran\AppData\Local\Continuum\anaconda3\lib\site-packages\sklearn\compose\_column_transformer.py", line 472, in fit_transform
    self._validate_transformers()
  File "C:\Users\Lofgran\AppData\Local\Continuum\anaconda3\lib\site-packages\sklearn\compose\_column_transformer.py", line 265, in _validate_transformers
    names, transformers, _ = zip(*self.transformers)
ValueError: not enough values to unpack (expected 3, got 2)
"""

The above exception was the direct cause of the following exception:

ValueError                                Traceback (most recent call last)
<ipython-input-17-6a051cf930ab> in <module>
     25 clf = GridSearchCV(pipe, param_grid=param_grid, cv=5, verbose=True, n_jobs=-1) #5 fold cross-validation; n_jobs=use all processors
     26 
---> 27 best_clf=clf.fit(X_train, y_train)

~\AppData\Local\Continuum\anaconda3\lib\site-packages\sklearn\model_selection\_search.py in fit(self, X, y, groups, **fit_params)
    686                 return results
    687 
--> 688             self._run_search(evaluate_candidates)
    689 
    690         # For multi-metric evaluation, store the best_index_, best_params_ and

~\AppData\Local\Continuum\anaconda3\lib\site-packages\sklearn\model_selection\_search.py in _run_search(self, evaluate_candidates)
   1147     def _run_search(self, evaluate_candidates):
   1148         """Search all candidates in param_grid"""
-> 1149         evaluate_candidates(ParameterGrid(self.param_grid))
   1150 
   1151 

~\AppData\Local\Continuum\anaconda3\lib\site-packages\sklearn\model_selection\_search.py in evaluate_candidates(candidate_params)
    665                                for parameters, (train, test)
    666                                in product(candidate_params,
--> 667                                           cv.split(X, y, groups)))
    668 
    669                 if len(out) < 1:

~\AppData\Local\Continuum\anaconda3\lib\site-packages\joblib\parallel.py in __call__(self, iterable)
    932 
    933             with self._backend.retrieval_context():
--> 934                 self.retrieve()
    935             # Make sure that we get a last message telling us we are done
    936             elapsed_time = time.time() - self._start_time

~\AppData\Local\Continuum\anaconda3\lib\site-packages\joblib\parallel.py in retrieve(self)
    831             try:
    832                 if getattr(self._backend, 'supports_timeout', False):
--> 833                     self._output.extend(job.get(timeout=self.timeout))
    834                 else:
    835                     self._output.extend(job.get())

~\AppData\Local\Continuum\anaconda3\lib\site-packages\joblib\_parallel_backends.py in wrap_future_result(future, timeout)
    519         AsyncResults.get from multiprocessing."""
    520         try:
--> 521             return future.result(timeout=timeout)
    522         except LokyTimeoutError:
    523             raise TimeoutError()

~\AppData\Local\Continuum\anaconda3\lib\concurrent\futures\_base.py in result(self, timeout)
    433                 raise CancelledError()
    434             elif self._state == FINISHED:
--> 435                 return self.__get_result()
    436             else:
    437                 raise TimeoutError()

~\AppData\Local\Continuum\anaconda3\lib\concurrent\futures\_base.py in __get_result(self)
    382     def __get_result(self):
    383         if self._exception:
--> 384             raise self._exception
    385         else:
    386             return self._result

ValueError: not enough values to unpack (expected 3, got 2)

编辑 2：

col_transform = ColumnTransformer(transformers=[ 
                                                ('imputer', SimpleImputer(missing_values=np.nan, strategy='median')),
                                                ('one_hot', OneHotEncoder(categories='auto', sparse=False))], 
                                  remainder='passthrough')

【问题讨论】：

您可以按原样发布错误消息吗？它指的是哪条线？
已按要求发布完整错误。简短回答：它指的是第 27 行（.fit 行）
您的代码至少有 2 个问题：(1) ('one_hot', OneHotEncoder(categories='auto', sparse=False), ['Sex', 'Embarked', 'Title'])。 OHE 不适用于列列表，您应该一一传递列。 (2) ('imputer', SimpleImputer(strategy='median'))，这是您的错误消息触发的地方，因为您的元组为 2，而 sklearn 期望元组为 3（在此处添加列列表）。真正的问题是即使纠正了错误，您的代码也不会运行。 ColumnTransformer 简直是耻辱。
@SergeyBushmanov - 我的列名列表是应该被编码的，而另一个（未列出的）列需要被估算。也许那里有一些误解。无视这一点，你的评论是有道理的。我将您提到的代码更改为如上述问题中的 Edit 2 所示，但是，我收到一个错误：ValueError: Found unknown categories ['Don'] in column 2 during transform（引用与上述相同的.fit() 行。'Don ' 是我的“标题”列中的一个值。所以我猜标题列要么没有正确编码，要么没有删除未编码的值？
您想要对列列表进行 OHE 编码的事实不算数... OHE 不接受列列表，您必须逐列进行。仔细阅读文档或查看讨论 here。您的另一个错误是由于 OHE 在训练和测试中看到不同的标记。相应地调整参数。即使在那之后ColumnTransformer 也不起作用....

标签： python-3.x scikit-learn

【解决方案1】：

对于那些遇到相同问题的人，这是我的发现：ColumnTransformer 中的转换器语法至关重要。它们必须是数字，对于OneHotEncoder，它显然必须采用slice(x, y,...z) 格式，参数是整数列号（我尝试将其添加为列表和元组，但均未成功）。谢尔盖·布什马诺夫（Sergey Bushmanov）的第一点是正确的（因为我没有正确指定列，所以我的估算器无法正常工作，但是没有迹象表明应该如何解决 OneHotEncoder 问题，所以我在标记这个回答正确。

ct = ColumnTransformer(
                       [("imputer", SimpleImputer(missing_values=np.nan, strategy='median'), [2]),
                        ("one_hot", OneHotEncoder(categories='auto', sparse=False), slice(1, 6, 7))])

【讨论】：