Sagemaker：DeepAR 超参数调整错误

Question

尝试初始化超参数调整作业时，在 SageMaker 上调整 DeepAR 时遇到新问题 - 调用 test:mean_wQuantileLoss 时也会出现此错误。我已经升级了 sagemaker 软件包，重新启动了我的实例，重新启动了内核（使用 juptyer 笔记本），但问题仍然存在。

ClientError: An error occurred (ValidationException) when calling the 
CreateHyperParameterTuningJob operation: The objective metric type, [Maximize], that you specified for objective metric, [test:RMSE], isn’t valid for the [156387875391.dkr.ecr.us-west-2.amazonaws.com/forecasting-deepar:1] algorithm. Choose a valid objective metric type.

代码：

my_tuner = HyperparameterTuner(estimator=estimator,
                               objective_metric_name="test:RMSE",
                               hyperparameter_ranges=hyperparams,
                               max_jobs=20,
                               max_parallel_jobs=2)

# Start hyperparameter tuning job
my_tuner.fit(inputs=data_channels)

Stack Trace:
ClientError                               Traceback (most recent call last)
<ipython-input-66-9d6d8de89536> in <module>()
      7 
      8 # Start hyperparameter tuning job
----> 9 my_tuner.fit(inputs=data_channels)

~/anaconda3/envs/python3/lib/python3.6/site-packages/sagemaker/tuner.py in fit(self, inputs, job_name, include_cls_metadata, **kwargs)
    255 
    256         self._prepare_for_training(job_name=job_name, include_cls_metadata=include_cls_metadata)
--> 257         self.latest_tuning_job = _TuningJob.start_new(self, inputs)
    258 
    259     @classmethod

~/anaconda3/envs/python3/lib/python3.6/site-packages/sagemaker/tuner.py in start_new(cls, tuner, inputs)
    525                                                output_config=(config['output_config']),
    526                                                resource_config=(config['resource_config']),
--> 527                                                stop_condition=(config['stop_condition']), tags=tuner.tags)
    528 
    529         return cls(tuner.sagemaker_session, tuner._current_job_name)

~/anaconda3/envs/python3/lib/python3.6/site-packages/sagemaker/session.py in tune(self, job_name, strategy, objective_type, objective_metric_name, max_jobs, max_parallel_jobs, parameter_ranges, static_hyperparameters, image, input_mode, metric_definitions, role, input_config, output_config, resource_config, stop_condition, tags)
    348         LOGGER.info('Creating hyperparameter tuning job with name: {}'.format(job_name))
    349         LOGGER.debug('tune request: {}'.format(json.dumps(tune_request, indent=4)))
--> 350         self.sagemaker_client.create_hyper_parameter_tuning_job(**tune_request)
    351 
    352     def stop_tuning_job(self, name):

~/anaconda3/envs/python3/lib/python3.6/site-packages/botocore/client.py in _api_call(self, *args, **kwargs)
    312                     "%s() only accepts keyword arguments." % py_operation_name)
    313             # The "self" in this scope is referring to the BaseClient.
--> 314             return self._make_api_call(operation_name, kwargs)
    315 
    316         _api_call.__name__ = str(py_operation_name)

~/anaconda3/envs/python3/lib/python3.6/site-packages/botocore/client.py in _make_api_call(self, operation_name, api_params)
    610             error_code = parsed_response.get("Error", {}).get("Code")
    611             error_class = self.exceptions.from_code(error_code)
--> 612             raise error_class(parsed_response, operation_name)
    613         else:
    614             return parsed_response

ClientError: An error occurred (ValidationException) when calling the CreateHyperParameterTuningJob operation: 
The objective metric type, [Maximize], that you specified for objective metric, [test:RMSE], isn’t valid for the [156387875391.dkr.ecr.us-west-2.amazonaws.com/forecasting-deepar:1] algorithm. 
Choose a valid objective metric type.

Answer 1

您似乎正试图通过 SageMaker HyperParameter Tuning 最大化该指标 test:RMSE can only be minimized。

要在 SageMaker Python SDK 中实现此目的，请使用 objective_type='Minimize' 创建 HyperparameterTuner。可以看到init方法here的签名。

以下是您应该对调用 HyperparameterTuner 进行的更改：

my_tuner = HyperparameterTuner(estimator=estimator,
                               objective_metric_name="test:RMSE",
                               objective_type='Minimize',
                               hyperparameter_ranges=hyperparams,
                               max_jobs=20,
                               max_parallel_jobs=2)