使用 AWS Lambda 和 api-gateway (python) 创建 API 时出错

Question

TL;DR：使用 AWS Lambda 和 API-Gateway 遇到错误

背景

我编写了一个 python 脚本来帮助我跟踪我正在使用的一些指数基金。它基本上是从不同网站上抓取（使用请求和 bs4）价格。

我想把它变成一个 API，这样我就可以在任何地方使用它，所以我使用 Flask 和 Connexion 来制作它（关注这个 https://realpython.com/flask-connexion-rest-api/）。这一切在本地都运行良好，但我在 AWS 上实施时遇到了麻烦。

我已将我的代码和相应的应用程序包压缩到 AWS Lambda 并在 API Gateway 中使用。但是我收到以下错误：

Endpoint response body before transformations: {"errorMessage": "module initialization error"} Method response body after transformations: {"errorMessage": "module initialization error"}

我已经在谷歌上搜索过，按照此处的步骤 (Amazon API Gateway : response body is not transformed when the API is called via Postman?)，部署了应用程序等 - 不知道为什么会这样。

这是一个sn-p的代码：

import requests
from bs4 import BeautifulSoup
from datetime import date, timedelta
import datetime
import json


def CurrentPortfolioValue(event, context):
    initial_porfolio_price = float (100)
    initial_date = date(2018, 8, 19) #set the initial date to a date where the fortnightly payment has gone
    num_of_shares_vanguard = float(500)
    num_of_shares_AMPNZShares = float(300)
    num_of_shares_AMPAustralisianProperty = float(100)
    fortnightly_contrib = float(100)
    current_value = initial_porfolio_price
    #print(initial_date)

    #sets a user-agent    
    example_headers = {'User-Agent':'Mozilla/5.0 (Windows NT 10.0; WOW64; rv:50.0) Gecko/20100101 Firefox/50.0'} #can make this better by having a list of random header


    #vanguard
    vanguard = requests.get("https://www.bloomberg.com/quote/VAN1579:AU",headers=example_headers) #need to have headers to avoid being blocked
    soup = BeautifulSoup(vanguard.content, 'html.parser')
    vanguard_price = soup.find_all("span", class_ = 'priceText__1853e8a5')[0].get_text()
    #print("Vanguard Price:\n" + vanguard_price+"AUD")

    #AMP Capital NZ Index 
    ampnzshares = requests.get("https://www.ampcapital.com/nz/en/investments/funds/index-funds/nz-shares-index-fund",headers=example_headers) #need to have headers to avoid being blocked
    soup1 = BeautifulSoup(ampnzshares.content, 'html.parser')
    ampnzshares_price = soup1.find_all("div", class_ = 'ht-module_title ht-highlight_color')[1].get_text()
    #print("AMP Capital NZ Index Price:\n" + ampnzshares_price+"NZD")

    #AMP Capital Australisian Property Index Fund
    ampAustralisianProperty = requests.get("https://www.ampcapital.com/nz/en/investments/funds/index-funds/australasian-property-index-fund",headers=example_headers) #need to have headers to avoid being blocked
    soup2 = BeautifulSoup(ampAustralisianProperty.content, 'html.parser')
    ampAustralianProperty_price = soup2.find_all("div", class_ = 'ht-module_title ht-highlight_color')[1].get_text()
    #print("AMP Capital Australisian Property Index Fund:\n" + ampAustralianProperty_price+"NZD")

    #get day difference between today and initial date
    current_date = date.today()
    days_difference = (current_date - initial_date).days #.days shows just the number of days
    #print(days_difference)

    count = days_difference
    while count > 14: #while difference is less than 14 days i.e forntightly payment hasnt been made
        current_value += initial_porfolio_price + (float(50) * float(vanguard_price)) + (float(50) * float(ampnzshares_price))
        count = count - 14
    print("Current Portfolio value is $" + str(current_value))
    return {
        "statusCode": 200,
        "body": json.dumps(current_value)}

CurrentPortfolioValue() #retrieves current portfolio value

我的 yaml 文件看起来像这样：

swagger: "2.0"
info:
  description: This is the swagger file that goes with our server code
  version: "1.0.0"
  title: Swagger REST Article
consumes:
  - "application/json"
produces:
  - "application/json"

basePath: "/api"

paths:
  /value: #name of the endpoint
    get:
      operationId: "ShareScraper.CurrentPortfolioValue" #link to the module - this is called when you navigate to /value
      tags:
        - "Portfolio"
      summary: "The portfolio data structure supported by the server application"
      description: "Read the current portfolio value"
      responses:
        200:
          description: "Successful read current portfolio value"
          schema:
            type: "string"
            items:
              properties:
                portfolio_value:
                  type: "string"

提前致谢！

[编辑] 这是我第一次创建 api 并使用 AWS，如果我遗漏了一些相当简单/明显的东西，非常抱歉

Answer 1

您的 Lambda 函数似乎没有正确格式化为 Lambda。根据 AWS 文档 here，您的 Lambda 必须格式化为处理程序，并且必须接受事件和上下文参数。

您需要编辑您的代码使其看起来更像这样才能运行：

from botocore.vendored import requests
from bs4 import BeautifulSoup
import datetime
from datetime import date, timedelta
import json


def lambda_handler(event, context):

    initial_porfolio_price = float (100)
    initial_date = date(2018, 8, 19) #set the initial date to a date where the fortnightly payment has gone
    num_of_shares_vanguard = float(500)
    num_of_shares_AMPNZShares = float(300)
    num_of_shares_AMPAustralisianProperty = float(100)
    fortnightly_contrib = float(100)
    current_value = initial_porfolio_price
    #print(initial_date)

    #sets a user-agent
    example_headers = {'User-Agent':'Mozilla/5.0 (Windows NT 10.0; WOW64; rv:50.0) Gecko/20100101 Firefox/50.0'} #can make this better by having a list of random header

    #vanguard
    vanguard = requests.get("https://www.bloomberg.com/quote/VAN1579:AU",headers=example_headers) #need to have headers to avoid being blocked
    soup = BeautifulSoup(vanguard.content, 'html.parser')
    print(soup)
    vanguard_price = soup.find_all("span", class_ = 'priceText__1853e8a5')[0].get_text()
    #print("Vanguard Price:\n" + vanguard_price+"AUD")

    #AMP Capital NZ Index
    ampnzshares = requests.get("https://www.ampcapital.com/nz/en/investments/funds/index-funds/nz-shares-index-fund",headers=example_headers) #need to have headers to avoid being blocked
    soup1 = BeautifulSoup(ampnzshares.content, 'html.parser')
    ampnzshares_price = soup1.find_all("div", class_ = 'ht-module_title ht-highlight_color')[1].get_text()
    #print("AMP Capital NZ Index Price:\n" + ampnzshares_price+"NZD")

    #AMP Capital Australisian Property Index Fund
    ampAustralisianProperty = requests.get("https://www.ampcapital.com/nz/en/investments/funds/index-funds/australasian-property-index-fund",headers=example_headers) #need to have headers to avoid being blocked
    soup2 = BeautifulSoup(ampAustralisianProperty.content, 'html.parser')
    ampAustralianProperty_price = soup2.find_all("div", class_ = 'ht-module_title ht-highlight_color')[1].get_text()
    #print("AMP Capital Australisian Property Index Fund:\n" + ampAustralianProperty_price+"NZD")

    #get day difference between today and initial date
    current_date = date.today()
    days_difference = (current_date - initial_date).days #.days shows just the number of days
    #print(days_difference)

    count = days_difference
    while count > 14: #while difference is less than 14 days i.e forntightly payment hasnt been made
        current_value += initial_porfolio_price + (float(50) * float(vanguard_price)) + (float(50) * float(ampnzshares_price))
    count = count - 14
    print("Current Portfolio value is $" + str(current_value))
    return {
       "statusCode": 200,
       "body": json.dumps(current_value)}

您没有发布导入语句，但至少您需要导入 json。

请注意，对函数的调用也被删除。我假设这个函数是独立运行的（即没有其他任何东西与它一起运行），因此您不需要在 Lambda 本身内调用它，因为它将在 Lambda 环境中执行。

编辑：

我已更新代码块并运行它。它确实在 Lambda 中运行，但是，在第 27 行，它试图找到在对 vanguard = requests.get... 请求的响应中不存在的 "span", class_ = 'priceText__1853e8a5'。相反，正如您在下面的屏幕截图中看到的那样，我们收到了其他信息，而 Lambda 向我们提供了错误 "errorMessage": "list index out of range"。

要找出原因，我们必须查看日志输出。当我们查看输出时，我们看到的不是一个包含我们试图抓取的定价信息的页面，而是接收到的实际上是一个验证码页面；彭博社认识到您正在抓取数据并且他们不希望人们抓取他们的数据，因此为了防止这种情况，他们希望您完成验证码，并且由于尚未建立处理这种情况的方法，因此 Lambda 出错了。

Answer 2

您的 lambda 代码没有 event 和 context 参数。 Which are required arguments for a lambda python handler.你可以修改你的代码，

def CurrentPortfolioValue(event, context):

    current_value += initial_porfolio_price + (float(50) * float(vanguard_price)) + (float(50) * float(ampnzshares_price))
    count = count - 14
    print("Current Portfolio value is $" + str(current_value))
    return {
        "statusCode": 200,
        "body": json.dumps(current_value)}

CurrentPortfolioValue() #retrieves current portfolio value