【问题标题】:ML Engine BigQuery: Request had insufficient authentication scopesML Engine BigQuery:请求的身份验证范围不足
【发布时间】:2017-09-13 01:04:28
【问题描述】:

我正在运行一个张量流模型,在 ml 引擎 上提交训练。我已经构建了一个 管道,它使用 tf.contrib.cloud.python.ops.bigquery_reader_ops.BigQueryReader 作为阅读器从 BigQuery 读取队列

使用 DataLab 和在 local 中一切正常,将 GOOGLE_APPLICATION_CREDENTIALS 变量设置为指向凭证密钥的 json 文件。但是,当我在云中提交训练作业时,我收到了这些错误(我只发布了主要的两个):

  1. 权限被拒绝:读取...的架构时出错(HTTP 响应代码 403,错误代码 0,错误消息“”)

  2. 创建模型时出错。检查详细信息:请求的身份验证范围不足。

我已经检查了其他所有内容,例如在脚本和项目/数据集/表 ID/名称中正确定义表架构

为了更清楚,我将日志中出现的整个错误粘贴在这里:

消息:“回溯(最近一次通话最后一次):

File "/usr/lib/python2.7/runpy.py", line 174, in _run_module_as_main
    "__main__", fname, loader, pkg_name)

File "/usr/lib/python2.7/runpy.py", line 72, in _run_code
    exec code in run_globals

File "/root/.local/lib/python2.7/site-packages/trainer/task.py", line 131, in <module>
    hparams=hparam.HParams(**args.__dict__)

File "/usr/local/lib/python2.7/dist-packages/tensorflow/contrib/learn/python/learn/learn_runner.py", line 210, in run
    return _execute_schedule(experiment, schedule)

File "/usr/local/lib/python2.7/dist-packages/tensorflow/contrib/learn/python/learn/learn_runner.py", line 47, in _execute_schedule
    return task()

 File "/usr/local/lib/python2.7/dist-packages/tensorflow/contrib/learn/python/learn/experiment.py", line 495, in train_and_evaluate
    self.train(delay_secs=0)

File "/usr/local/lib/python2.7/dist-packages/tensorflow/contrib/learn/python/learn/experiment.py", line 275, in train
    hooks=self._train_monitors + extra_hooks)

File "/usr/local/lib/python2.7/dist-packages/tensorflow/contrib/learn/python/learn/experiment.py", line 665, in _call_train
    monitors=hooks)

File "/usr/local/lib/python2.7/dist-packages/tensorflow/python/util/deprecation.py", line 289, in new_func
    return func(*args, **kwargs)

File "/usr/local/lib/python2.7/dist-packages/tensorflow/contrib/learn/python/learn/estimators/estimator.py", line 455, in fit
    loss = self._train_model(input_fn=input_fn, hooks=hooks)

File "/usr/local/lib/python2.7/dist-packages/tensorflow/contrib/learn/python/learn/estimators/estimator.py", line 1007, in _train_model
    _, loss = mon_sess.run([model_fn_ops.train_op, model_fn_ops.loss])

File "/usr/local/lib/python2.7/dist-packages/tensorflow/python/training/monitored_session.py", line 521, in __exit__
    self._close_internal(exception_type)

File "/usr/local/lib/python2.7/dist-packages/tensorflow/python/training/monitored_session.py", line 556, in _close_internal
    self._sess.close()

File "/usr/local/lib/python2.7/dist-packages/tensorflow/python/training/monitored_session.py", line 791, in close
    self._sess.close()

File "/usr/local/lib/python2.7/dist-packages/tensorflow/python/training/monitored_session.py", line 888, in close
    ignore_live_threads=True)

File "/usr/local/lib/python2.7/dist-packages/tensorflow/python/training/coordinator.py", line 389, in join
    six.reraise(*self._exc_info_to_raise)

File "/usr/local/lib/python2.7/dist-packages/tensorflow/python/training/queue_runner_impl.py", line 238, in _run
    enqueue_callable()

File "/usr/local/lib/python2.7/dist-packages/tensorflow/python/client/session.py", line 1063, in _single_operation_run
    target_list_as_strings, status, None)

File "/usr/lib/python2.7/contextlib.py", line 24, in __exit__
    self.gen.next()

File "/usr/local/lib/python2.7/dist-packages/tensorflow/python/framework/errors_impl.py", line 466, in raise_exception_on_not_ok_status
    pywrap_tensorflow.TF_GetCode(status))
PermissionDeniedError: Error executing an HTTP request (HTTP response code 403, error code 0, error message '')
     when reading schema for pasquinelli-bigdata:Transactions.t_11_Hotel_25_w_train@1505224768418
     [[Node: GenerateBigQueryReaderPartitions = GenerateBigQueryReaderPartitions[columns=["F_RACC_GEST", "LABEL", "F_RCA", "W24", "ETA", "W22", "W23", "W20", "W21", "F_LEASING", "W2", "W16", "WLABEL", "SEX", "F_PIVA", "F_MUTUO", "Id_client", "F_ASS_VITA", "F_ASS_DANNI", "W19", "W18", "W17", "PROV", "W15", "W14", "W13", "W12", "W11", "W10", "W7", "W6", "W5", "W4", "W3", "F_FIN", "W1", "ImpTot", "F_MULTIB", "W9", "W8"], dataset_id="Transactions", num_partitions=1, project_id="pasquinelli-bigdata", table_id="t_11_Hotel_25_w_train", test_end_point="", timestamp_millis=1505224768418, _device="/job:localhost/replica:0/task:0/cpu:0"]()]]

任何建议都会非常有帮助,因为我对 GC 比较陌生。 谢谢大家。

【问题讨论】:

    标签: authentication google-bigquery google-cloud-ml-engine


    【解决方案1】:

    对从 Cloud ML Engine 读取 BigQuery 数据的支持仍在开发中,因此目前不支持您正在执行的操作。您遇到的问题是 ML Engine 运行的机器没有与 BigQuery 对话的正确范围。您在本地运行时可能还会遇到的一个潜在问题是 BigQuery 读取性能不佳。这是需要解决的两个工作示例。

    同时,我建议将数据导出到 GCS 进行训练。这将更具可扩展性,因此随着数据的增加,您不必担心训练性能不佳。这可能是一个很好的模式,它可以让您对数据进行一次预处理,将结果以 CSV 格式写入 GCS,然后进行多次训练运行以尝试不同的算法或超参数。

    【讨论】:

    • 非常感谢您的回答。我会按照你的建议进行。
    猜你喜欢
    • 2017-10-12
    • 2017-08-09
    • 2021-08-07
    • 2021-01-04
    • 2021-10-20
    • 2021-10-16
    • 1970-01-01
    • 1970-01-01
    • 1970-01-01
    相关资源
    最近更新 更多