使用无服务器框架部署 AWS Lambda 时缺少依赖项

Question

我正在尝试使用无服务器框架将预测 Lambda 处理程序部署到 AWS，我的代码依赖项是 pytorch 和 transformers(HF)。

我正面临这个问题，不知道如何克服，

我已经正确配置了 requirements.txt 文件，它有以下两行，

transformers==2.10
https://download.pytorch.org/whl/cpu/torch-1.5.0%2Bcpu-cp38-cp38-linux_x86_64.whl

当我尝试使用“无服务器部署”命令时，有时它会创建一个包，其中包含包内 requirements.zip 中的所有依赖项，大约 144 MB，这是预期的结果。

日志供您参考

C:\Users\Rajeshwaranp\serverless-HFT>sls deploy
Serverless: Deprecation warning: Starting with next major version, API Gateway naming will be changed from "{stage}-{service}" to "{service}-{stage}".
            Set "provider.apiGateway.shouldStartNameWithService" to "true" to adapt to the new behavior now.
            More Info: https://www.serverless.com/framework/docs/deprecations/#AWS_API_GATEWAY_NAME_STARTING_WITH_SERVICE
Serverless: Adding Python requirements helper...
Serverless: Generated requirements from C:\Users\Rajeshwaranp\serverless-HFT\requirements.txt in C:\Users\Rajeshwaranp\serverless-HFT\.serverless\requirements.txt...
Serverless: Installing requirements from C:\Users\Rajeshwaranp\serverless-HFT\cache\72130b8b1c12a9cc09719c938b787b829ceb78a73eeafa251ccf974d7dafec58_slspyc\requirements.txt ...
Serverless: Docker Image: lambci/lambda:build-python3.8
Serverless: Using download cache directory C:\Users\Rajeshwaranp\serverless-HFT\cache\downloadCacheslspyc
Serverless: Running docker run --rm -v C\:/Users/Rajeshwaranp/serverless-HFT/cache/72130b8b1c12a9cc09719c938b787b829ceb78a73eeafa251ccf974d7dafec58_slspyc\:/var/task\:z -v C\:/Users/Rajeshwaranp/serverless-HFT/cache/downloadCacheslspyc\:/var/useDownloadCache\:z -u 0 lambci/lambda\:build-python3.8 python -m pip install -t /var/task/ -r /var/task/requirements.txt --cache-dir /var/useDownloadCache...
Serverless: Zipping required Python packages...
Serverless: Packaging service...
Serverless: Excluding development dependencies...
Serverless: Removing Python requirements helper...
Serverless: Injecting required Python packages to package...
Serverless: WARNING: Function predict_answer has timeout of 60 seconds, however, it's attached to API Gateway so it's automatically limited to 30 seconds.
Serverless: Creating Stack...
Serverless: Checking Stack create progress...
........
Serverless: Stack create finished...
Serverless: Uploading CloudFormation file to S3...
Serverless: Uploading artifacts...
Serverless: Uploading service serverless-hft.zip file to S3 (144.02 MB)...
Serverless: Validating template...
Serverless: Updating Stack...
Serverless: Checking Stack update progress...
.................................
Serverless: Stack update finished...

但有时，当我使用相同的命令进行部署时，它会创建一个 128KB（大约）的包，而包内的 requirements.zip 文件中没有任何依赖项。

日志供您参考，

C:\Users\Rajeshwaranp\serverless-HFT>sls deploy
Serverless: Deprecation warning: Starting with next major version, API Gateway naming will be changed from "{stage}-{service}" to "{service}-{stage}".
            Set "provider.apiGateway.shouldStartNameWithService" to "true" to adapt to the new behavior now.
            More Info: https://www.serverless.com/framework/docs/deprecations/#AWS_API_GATEWAY_NAME_STARTING_WITH_SERVICE
Serverless: Adding Python requirements helper...
Serverless: Generated requirements from C:\Users\Rajeshwaranp\serverless-HFT\requirements.txt in C:\Users\Rajeshwaranp\serverless-HFT\.serverless\requirements.txt...
Serverless: Using static cache of requirements found at C:\Users\Rajeshwaranp\serverless-HFT\cache\72130b8b1c12a9cc09719c938b787b829ceb78a73eeafa251ccf974d7dafec58_slspyc ...
Serverless: Zipping required Python packages...
Serverless: Packaging service...
Serverless: Excluding development dependencies...
Serverless: Removing Python requirements helper...
Serverless: Injecting required Python packages to package...
Serverless: WARNING: Function predict_answer has timeout of 60 seconds, however, it's attached to API Gateway so it's automatically limited to 30 seconds.
Serverless: Uploading CloudFormation file to S3...
Serverless: Uploading artifacts...
Serverless: Uploading service serverless-hft.zip file to S3 (124.15 KB)...
Serverless: Validating template...
Serverless: Updating Stack...
Serverless: Checking Stack update progress...
..............
Serverless: Stack update finished...

那么为什么会这样呢？以及如何调整部署命令以便始终包含依赖项？

Answer 1

使用 lambda 层打包您的所有需求，确保您在 requirements.txt 文件中包含所有需求。

这仅在插件部分中列出 serverless-python-requirements 插件时有效。 用这个替换您的自定义键，并为需要这些要求的函数提供使用该层的参考。

plugins:
  - serverless-python-requirements
custom:
  pythonRequirements:
    layer: true
functions:
  auth:
    handler: data.auth
    layers:
      - { Ref: PythonRequirementsLambdaLayer}