“GBTClassificationModel”对象没有属性“fitMultiple”-pyspark答案

【问题标题】：'GBTClassificationModel' object has no attribute 'fitMultiple' - pyspark“GBTClassificationModel”对象没有属性“fitMultiple”-pyspark
【发布时间】：2019-10-30 09:30:03
【问题描述】：

尝试对 GBT 进行交叉验证时遇到以下错误消息。我之前运行 GBT 模型没有任何问题。

不再支持 fitMultiple 吗？顺便说一句，我正在使用 PySpark 2.4.4。

这是我的代码：

from pyspark.ml.tuning import ParamGridBuilder, CrossValidator

paramGrid = ParamGridBuilder().addGrid(gbt.maxDepth, [2, 4, 6]).addGrid(gbt.maxBins, [20, 60]).addGrid(gbt.maxIter, [10, 20]).build()
evaluator = BinaryClassificationEvaluator()
cv = CrossValidator(estimator=gbt, estimatorParamMaps=paramGrid, evaluator=evaluator)

# Run cross validations.  This can take about 6 minutes since it is training over 20 trees!
cvModel = cv.fit(train)
predictions = cvModel.transform(test)
evaluator.evaluate(predictions)

这是错误信息

---------------------------------------------------------------------------
AttributeError                            Traceback (most recent call last)
<ipython-input-150-95e5a7d270fe> in <module>
      6 
      7 # Run cross validations.  This can take about 6 minutes since it is training over 20 trees!
----> 8 cvModel = cv.fit(train)
      9 predictions = cvModel.transform(test)
     10 evaluator.evaluate(predictions)

/anaconda_env/personal/gleow/py37/lib/python3.7/site-packages/pyspark/ml/base.py in fit(self, dataset, params)
    130                 return self.copy(params)._fit(dataset)
    131             else:
--> 132                 return self._fit(dataset)
    133         else:
    134             raise ValueError("Params must be either a param map or a list/tuple of param maps, "

/anaconda_env/personal/gleow/py37/lib/python3.7/site-packages/pyspark/ml/tuning.py in _fit(self, dataset)
    301             train = df.filter(~condition).cache()
    302 
--> 303             tasks = _parallelFitTasks(est, train, eva, validation, epm, collectSubModelsParam)
    304             for j, metric, subModel in pool.imap_unordered(lambda f: f(), tasks):
    305                 metrics[j] += (metric / nFolds)

/anaconda_env/personal/gleow/py37/lib/python3.7/site-packages/pyspark/ml/tuning.py in _parallelFitTasks(est, train, eva, validation, epm, collectSubModel)
     47     :return: (int, float, subModel), an index into `epm` and the associated metric value.
     48     """
---> 49     modelIter = est.fitMultiple(train, epm)
     50 
     51     def singleTask():

AttributeError: 'GBTClassificationModel' object has no attribute 'fitMultiple'

【问题讨论】：

您在谈论 GBT，而您的代码显示随机森林 - 请相应地进行编辑
已编辑。我想用随机森林试一试，但效果不佳。

标签： python machine-learning pyspark apache-spark-ml

【解决方案1】：

对我的代码进行故障排除并找到错误...原来我已经初始化了 gbt 并安装了以下内容：

gbt = GBTClassifier(maxIter=10)
gbt = gbt.fit(train)
...
...

所以在尝试运行以下内容时：

from pyspark.ml.tuning import ParamGridBuilder, CrossValidator

paramGrid = ...
evaluator = ...
cv = CrossValidator(estimator=gbt, estimatorParamMaps=paramGrid, evaluator=evaluator)

CrossValidator 实际上正在调用“gbt.fit(train)”，它不再是分类器对象。所以我所做的就是再次初始化 gbt = GBTClassifier(maxIter=10)。

【讨论】：