【发布时间】:2020-02-15 12:29:41
【问题描述】:
我有以下代码
client = bigquery.Client()
dataset_id = 'dataset' # replace with your dataset ID
table_id = 'table' # replace with your table ID
table_ref = client.dataset(dataset_id).table(table_id)
table = client.get_table(table_ref) # API request
rows_to_insert = []
bq = bigquery.Client(project='project-id')
query = """SELECT Url FROM `project-id.dataset.urltable`"""
query_job = bq.query(query)
data = query_job.result()
rows = list(data)
def main():
for row in rows:
URL = urllib.request.urlopen(row[0])
soup_page = soup(URL, features="lxml")
try:
data = json.loads(soup_page.find_all('script', type='application/ld+json')[1].text)
except:
data ='unknown'
try:
price_ruw = data['offers']['price']
shopprice = price_ruw.replace(',','.')
except:
price = 0
try:
ean = data['gtin13']
ean = str(ean)
except:
ean = 'unknown'
try:
title_ruw1 = data['name']
title_ruw = title_ruw1
tile_trim = title_ruw[:750]
title = tile_trim.replace("'", "")
except:
title = "unknown"
try:
reviews = data['aggregateRating']['reviewCount']
except:
reviews = 0
try:
score = (float(data['aggregateRating']['ratingValue']) * 2)
except:
score = 0
datenow = (datetime.datetime.now())
shoplink = row[0]
rows_to_insert.append([shoplink,ean,title,reviews,score,shopprice,datenow])
client.insert_rows(table, rows_to_insert) # API request
main()
在谷歌云平台测试这段代码给出了
Error: function crashed. Details:
main() takes 0 positional arguments but 2 were given
但是,在部署此代码时,它不会出错。仅安排此查询不起作用,因为它不断给出以下错误。
为了部署,我使用以下命令(有效)
gcloud functions deploy <function> --entry-point main --
runtime python37 --trigger-resource <name> --trigger-event google.pubsub.topic.publish --timeout 540s
【问题讨论】:
-
您的
rows定义在哪里。这是完整的代码吗? -
@Frank 你能粘贴完整的堆栈跟踪吗?
-
除了未定义行和客户端之外,当我尝试调用它时,其他一切都运行良好。检查你是否没有两次调用一个函数,或者你是否没有用其他函数覆盖函数。
-
您是否在 Cloud Functions 中运行此代码?如果是,您需要使用入口点函数而不是
main()。还要发布您的所有代码,包括导入。显示你是如何运行这个程序的(命令行)。调试时上下文很重要。 -
感谢您的回复。我添加了更多上下文。 - 将 main() 更改为 function() 不起作用。部署时会报错:文件 main.py 应该包含一个名为 main 的函数。我删除了 --entry-point main
标签: python google-cloud-platform google-cloud-functions