h2o ensemble 抛出错误：“基本模型不保留交叉验证预测”

Question

我正在尝试根据许多 GLM、GBM 和深度学习模型在 H2O 中创建一个集成模型。

这是我到目前为止所做的。

导入相关库：

import h2o
from h2o.estimators.glm import H2OGeneralizedLinearEstimator
from h2o.estimators.gbm import H2OGradientBoostingEstimator
from h2o.estimators.deeplearning import H2ODeepLearningEstimator
from h2o.estimators.stackedensemble import H2OStackedEnsembleEstimator
from h2o.grid.grid_search import H2OGridSearch

数据可以从here下载。进口：

airlines = h2o.import_file(path = "/Users/alexwoolford/h2o/allyears2k.csv", destination_frame = "airlines.hex")

分成训练/测试集：

airlines_80,airlines_20 = airlines.split_frame(ratios=[.8], destination_frames=["airlines_80.hex", "airlines_20.hex"])

定义变量（将 y 预测为 x 中所有列的函数）：

x= airlines.columns
y= "ArrDelay"
x.remove(y)

设置常用属性：

folds=5
assignment_type="Modulo"
search_criteria={'strategy': 'RandomDiscrete', 'max_models': 5, 'seed': 1}

使用 H2O 的网格搜索创建各种模型：

# GLM
glm_params = {"alpha": [0. , 0.1, 0.2, 0.3, 0.4, 0.5, 0.6, 0.7, 0.8, 0.9, 1.],
              "lambda": [0, 1e-7, 1e-5, 1e-3, 1e-1]}

glm_grid = H2OGridSearch(model=H2OGeneralizedLinearEstimator(fold_assignment=assignment_type, nfolds=folds),
                         grid_id='glm_grid',
                         hyper_params=glm_params,
                         search_criteria=search_criteria)
glm_grid.train(x=x,
               y=y,
               training_frame=airlines_80,
               validation_frame=airlines_20)

# GBM
gbm_params = {'learn_rate': [0.01, 0.03],
              'max_depth': [3, 4, 5, 6, 9],
              'sample_rate': [0.7, 0.8, 0.9, 1],
              'col_sample_rate': [0.2, 0.3, 0.4, 0.5, 0.6, 0.7, 0.8]}

gbm_grid = H2OGridSearch(model=H2OGradientBoostingEstimator(fold_assignment=assignment_type, nfolds=folds),
                         grid_id='gbm_grid',
                         hyper_params=gbm_params,
                         search_criteria=search_criteria)
gbm_grid.train(x=x,
               y=y,
               training_frame=airlines_80,
               validation_frame=airlines_20)

# Deep learning
dl_params = {'activation': ['rectifier', 'rectifier_with_dropout'],
             'hidden': [[10,10], [20,15], [50,50,50]],
             'l1': [0, 1e-3, 1e-5],
             'l2': [0, 1e-3, 1e-5]}

dl_grid = H2OGridSearch(model=H2ODeepLearningEstimator(fold_assignment=assignment_type, nfolds=folds),
                        grid_id='dl_grid',
                        hyper_params=dl_params,
                        search_criteria=search_criteria)

dl_grid.train(x=x,
              y=y,
              training_frame=airlines_80,
              validation_frame=airlines_20)

获取所有model_id的列表：

all_model_ids = glm_grid.model_ids + gbm_grid.model_ids + dl_grid.model_ids

我尝试创建合奏的地方：

ensemble = H2OStackedEnsembleEstimator(base_models=all_model_ids)
ensemble.train(x=x, y=y, training_frame=airlines_80, validation_frame=airlines_20)

...抛出以下错误：

stackedensemble Model Build progress: | (failed)
---------------------------------------------------------------------------
OSError                                   Traceback (most recent call last)
<ipython-input-26-bc7b6094816f> in <module>()
      1 ensemble = H2OStackedEnsembleEstimator(base_models=all_model_ids)
----> 2 ensemble.train(x=x, y=y, training_frame=airlines_80, validation_frame=airlines_20)

/anaconda3/lib/python3.6/site-packages/h2o/estimators/estimator_base.py in train(self, x, y, training_frame, offset_column, fold_column, weights_column, validation_frame, max_runtime_secs, ignored_columns, model_id, verbose)
    235             return
    236 
--> 237         model.poll(verbose_model_scoring_history=verbose)
    238         model_json = h2o.api("GET /%d/Models/%s" % (rest_ver, model.dest_key))["models"][0]
    239         self._resolve_model(model.dest_key, model_json)

/anaconda3/lib/python3.6/site-packages/h2o/job.py in poll(self, verbose_model_scoring_history)
     75             if (isinstance(self.job, dict)) and ("stacktrace" in list(self.job)):
     76                 raise EnvironmentError("Job with key {} failed with an exception: {}\nstacktrace: "
---> 77                                        "\n{}".format(self.job_key, self.exception, self.job["stacktrace"]))
     78             else:
     79                 raise EnvironmentError("Job with key %s failed with an exception: %s" % (self.job_key, self.exception))

OSError: Job with key $03017f00000132d4ffffffff$_a2359a38ec8d31316aee91398f0249f8 failed with an exception: water.exceptions.H2OIllegalArgumentException: Base model does not keep cross-validation predictions: 5
stacktrace: 
water.exceptions.H2OIllegalArgumentException: Base model does not keep cross-validation predictions: 5
    at hex.StackedEnsembleModel.checkAndInheritModelProperties(StackedEnsembleModel.java:382)
    at hex.ensemble.StackedEnsemble$StackedEnsembleDriver.computeImpl(StackedEnsemble.java:234)
    at hex.ModelBuilder$Driver.compute2(ModelBuilder.java:218)
    at water.H2O$H2OCountedCompleter.compute(H2O.java:1395)
    at jsr166y.CountedCompleter.exec(CountedCompleter.java:468)
    at jsr166y.ForkJoinTask.doExec(ForkJoinTask.java:263)
    at jsr166y.ForkJoinPool$WorkQueue.runTask(ForkJoinPool.java:974)
    at jsr166y.ForkJoinPool.runWorker(ForkJoinPool.java:1477)
    at jsr166y.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:104)

你能看出我做错了什么吗？

Answer 1

您的每个模型中似乎都缺少参数keep_cross_validation_predictions=True。例如，您希望对 GLM 执行以下操作，然后对要堆叠的其他模型也执行以下操作：

glm_grid = H2OGridSearch(model=H2OGeneralizedLinearEstimator(fold_assignment=assignment_type, nfolds=folds,
    keep_cross_validation_predictions=True),
                                 grid_id='glm_grid',
                                 hyper_params=glm_params,
                                 search_criteria=search_criteria)