Google Cloud BigQuery 身份验证 TransportError 404答案

【问题标题】：Google Cloud BigQuery authentication TransportError 404Google Cloud BigQuery 身份验证 TransportError 404
【发布时间】：2020-04-27 19:25:26
【问题描述】：

这里是 GCP 新手。我有一个简单的 SQL 查询来堆叠溢出数据集：

SQL查询代码：

SELECT
  CONCAT(
    'https://stackoverflow.com/questions/',
    CAST(id as STRING)) as url,
  view_count
FROM `bigquery-public-data.stackoverflow.posts_questions`
WHERE tags like '%google-bigquery%'
ORDER BY view_count DESC
LIMIT 10

这在 Bigquery Web UI 中运行良好，所以我假设与项目设置、Bigiquery 启用等相关的任何内容都已排序，但是当我在 Compute 引擎中尝试此 Python 代码时：

from google.cloud import bigquery

client = bigquery.Client()

sql = '''
SELECT
  CONCAT(
    'https://stackoverflow.com/questions/',
    CAST(id as STRING)) as url,
  view_count
FROM `bigquery-public-data.stackoverflow.posts_questions`
WHERE tags like '%google-bigquery%'
ORDER BY view_count DESC
LIMIT 10
'''

query_job = client.query(sql)
df = query_job.to_dataframe()

我收到一个不太明确的 404 错误：

Traceback (most recent call last):
  File "/home/user/.local/lib/python3.5/site-packages/google/auth/compute_engine/credentials.py", line 96, in refresh
    self._retrieve_info(request)
  File "/home/user/.local/lib/python3.5/site-packages/google/auth/compute_engine/credentials.py", line 77, in _retrieve_info
    request, service_account=self._service_account_email
  File "/home/user/.local/lib/python3.5/site-packages/google/auth/compute_engine/_metadata.py", line 219, in get_service_account_info
    recursive=True,
  File "/home/user/.local/lib/python3.5/site-packages/google/auth/compute_engine/_metadata.py", line 172, in get
    response,
google.auth.exceptions.TransportError: ('Failed to retrieve http://metadata.google.internal/computeMetadata/v1/instance/service-accounts/default/?recursive=true from the Google Compute Enginemetadata service. Status: 404 Response:\nb\'<!DOCTYPE html>\\n<html lang=en>\\n  <meta charset=utf-8>\\n  <meta name=viewport content="initial-scale=1, minimum-scale=1, width=device-width">\\n  <title>Error 404 (Not Found)!!1</title>\\n  <style>\\n    *{margin:0;padding:0}html,code{font:15px/22px arial,sans-serif}html{background:#fff;color:#222;padding:15px}body{margin:7% auto 0;max-width:390px;min-height:180px;padding:30px 0 15px}* > body{background:url(//www.google.com/images/errors/robot.png) 100% 5px no-repeat;padding-right:205px}p{margin:11px 0 22px;overflow:hidden}ins{color:#777;text-decoration:none}a img{border:0}@media screen and (max-width:772px){body{background:none;margin-top:0;max-width:none;padding-right:0}}#logo{background:url(//www.google.com/images/branding/googlelogo/1x/googlelogo_color_150x54dp.png) no-repeat;margin-left:-5px}@media only screen and (min-resolution:192dpi){#logo{background:url(//www.google.com/images/branding/googlelogo/2x/googlelogo_color_150x54dp.png) no-repeat 0% 0%/100% 100%;-moz-border-image:url(//www.google.com/images/branding/googlelogo/2x/googlelogo_color_150x54dp.png) 0}}@media only screen and (-webkit-min-device-pixel-ratio:2){#logo{background:url(//www.google.com/images/branding/googlelogo/2x/googlelogo_color_150x54dp.png) no-repeat;-webkit-background-size:100% 100%}}#logo{display:inline-block;height:54px;width:150px}\\n  </style>\\n  <a href=//www.google.com/><span id=logo aria-label=Google></span></a>\\n  <p><b>404.</b> <ins>That\\xe2\\x80\\x99s an error.</ins>\\n  <p>The requested URL <code>/computeMetadata/v1/instance/service-accounts/default/?recursive=true</code> was not found on this server.  <ins>That\\xe2\\x80\\x99s all we know.</ins>\\n\'', <google.auth.transport.requests._Response object at 0x7fa2026a3710>)

我是否遗漏了从 Compute Engine 中启用身份验证的任何信息？从文档看来，从 Compute Engine 中运行时似乎不需要身份验证步骤。

【问题讨论】：

标签： google-cloud-platform google-bigquery

【解决方案1】：

正如您在official documentation 中看到的，使用google-cloud-bigquery 库需要四个步骤。其中之一是Setup Authentication。此外，请确保您的 Python 版本高于或等于 3.5。

在 Google Compute Engine 上运行时，默认凭据的一个常见问题是虚拟机没有足够的范围来查询 BigQuery。所以我建议使用服务帐户进行身份验证。

您还可以参考 BigQuery Client page，它展示了如何开始使用 BigQuery API 的云客户端库。

希望以上信息对您有用。

【讨论】：