【问题标题】:Google Cloud Function + bigquery: Internal Server ErrorGoogle Cloud Function + bigquery:内部服务器错误
【发布时间】:2021-04-05 06:30:22
【问题描述】:

我正在用 python 制作一个谷歌云函数,它向 API 请求数据,执行 ETL,最后将生成的 panda 的数据框放入一个大查询表中。

部署是正确的,但是当我触发函数(HTTP 触发器)时,我得到了这个错误:

Internal Server Error
The server encountered an internal error and was unable to complete your request. Either the server is overloaded or there is an error in the application.

知道我做错了什么吗???这是我的代码的简化版本:

import pandas as pd
from google.cloud import bigquery, error_reporting
from bigquery_tools import update_table

def main(request):

  if request:
    try:
        # BIGQUERY CLIENT
        BIGQUERY_CREDENTIALS = "credentials.json"
        BIGQUERY_PROJECT_ID = "my_project_id"
        BIGQUERY_DATASET_ID = "my_dataset_id"
        TABLE_ID = "my_table"
        CLIENT = bigquery.Client(project=BIGQUERY_PROJECT_ID)

        # SOME DATAFRAME
        df = pd.DataFrame({
            "debug": ["debug_a"]
        })
        
        # SAVE TO BIGQUERY
        try:
            dataset_ref = CLIENT.dataset(BIGQUERY_DATASET_ID)
            table_ref = dataset_ref.table(TABLE_ID)
            job_config = bigquery.LoadJobConfig()
            job_config.source_format = bigquery.SourceFormat.PARQUET
            job_config.autodetect = True
            job_config.write_disposition = bigquery.WriteDisposition.WRITE_TRUNCATE

            job = CLIENT.load_table_from_dataframe(
                df,
                table_ref,
                job_config=job_config
            )
            job.result()  # Waits for table load to complete.

        except Exception as e:
            pass
    except Exception as e:
        pass

【问题讨论】:

  • 您能否分享您的依赖版本、从 API 获得的数据大小以及您的 Cloud Function 设置(内存、区域...)?谢谢
  • 我解决了我的问题:问题是我忘记了返回语句('some message',http_code)。例如:return ('ok',200)

标签: python python-3.x google-bigquery google-cloud-functions


【解决方案1】:

[已解决]问题是我忘记了return ('some message', http_code)的声明。

例如:return ('ok',200)

        try:
            dataset_ref = CLIENT.dataset(BIGQUERY_DATASET_ID)
            table_ref = dataset_ref.table(TABLE_ID)
            job_config = bigquery.LoadJobConfig()
            job_config.source_format = bigquery.SourceFormat.PARQUET
            job_config.autodetect = True
            job_config.write_disposition = bigquery.WriteDisposition.WRITE_TRUNCATE

            job = CLIENT.load_table_from_dataframe(
                df,
                table_ref,
                job_config=job_config
            )
            job.result()  # Waits for table load to complete.
            return ("ok", 200)
        except Exception as e:
            return ("error", 400)

【讨论】:

    猜你喜欢
    • 1970-01-01
    • 2023-02-17
    • 2020-09-20
    • 2020-07-06
    • 1970-01-01
    • 1970-01-01
    • 1970-01-01
    • 1970-01-01
    相关资源
    最近更新 更多