触发与 EFS 关联的 Lambda 函数时未下载 S3 文件

Question

我正在使用无服务器框架创建一个 Lambda 函数，当由 S3 上传（将 test.vcf 上传到 s3://trigger-test/uploads/）触发时，将上传的文件从 S3 下载到 EFS（特别是到 /mnt/efs/vcfs/ 文件夹）。我对 EFS 很陌生，并遵循 AWS 文档设置 EFS 访问点，但是当我部署此应用程序并上传测试文件以触发 Lambda 函数时，它无法下载文件并在 CloudWatch 日志中给出此错误：

[ERROR] FileNotFoundError: [Errno 2] No such file or directory: '/mnt/efs/vcfs/test.vcf.A0bA45dC'
Traceback (most recent call last):
  File "/var/task/handler.py", line 21, in download_files_to_efs
    result = s3.download_file('trigger-test', key, efs_loci)
  File "/var/runtime/boto3/s3/inject.py", line 170, in download_file
    return transfer.download_file(
  File "/var/runtime/boto3/s3/transfer.py", line 307, in download_file
    future.result()
  File "/var/runtime/s3transfer/futures.py", line 106, in result
    return self._coordinator.result()
  File "/var/runtime/s3transfer/futures.py", line 265, in result
    raise self._exception
  File "/var/runtime/s3transfer/tasks.py", line 126, in __call__
    return self._execute_main(kwargs)
  File "/var/runtime/s3transfer/tasks.py", line 150, in _execute_main
    return_value = self._main(**kwargs)
  File "/var/runtime/s3transfer/download.py", line 571, in _main
    fileobj.seek(offset)
  File "/var/runtime/s3transfer/utils.py", line 367, in seek
    self._open_if_needed()
  File "/var/runtime/s3transfer/utils.py", line 350, in _open_if_needed
    self._fileobj = self._open_function(self._filename, self._mode)
  File "/var/runtime/s3transfer/utils.py", line 261, in open
    return open(filename, mode)

我的直觉是，这与 Lambda 函数中指定的本地挂载路径与 EFS 接入点配置的详细信息部分中的 Root directory path 有关。最终，我希望将上传到 S3 的 test.vcf 文件下载到 EFS 文件夹：/mnt/efs/vcfs/。

相关文件：

serverless.yml：

service: LambdaEFS-trigger-test
frameworkVersion: '2'

provider:
  name: aws
  runtime: python3.8
  stage: dev
  region: us-west-2
  vpc:
    securityGroupIds:
      - sg-XXXXXXXX
      - sg-XXXXXXXX
      - sg-XXXXXXXX
    subnetIds:
      - subnet-XXXXXXXXXX


functions:
  cfnPipelineTrigger:
    handler: handler.download_files_to_efs
    description: Lambda to download S3 file to EFS folder.
    events:
      - s3:
          bucket: trigger-test
          event: s3:ObjectCreated:*
          rules:
            - prefix: uploads/
            - suffix: .vcf
          existing: true
    fileSystemConfig:
      localMountPath: /mnt/efs
      arn: arn:aws:elasticfilesystem:us-west-2:XXXXXXXXXX:access-point/fsap-XXXXXXX
    iamRoleStatements:
      - Effect: Allow
        Action:
          - s3:ListBucket
        Resource:
          - arn:aws:s3:::trigger-test
      - Effect: Allow
        Action:
          - s3:GetObject
          - s3:GetObjectVersion
        Resource:
          - arn:aws:s3:::trigger-test/uploads/* 
      - Effect: Allow
        Action:
        - elasticfilesystem:ClientMount
        - elasticfilesystem:ClientWrite
        - elasticfilesystem:ClientRootAccess
        Resource:
        - arn:aws:elasticfilesystem:us-west-2:XXXXXXXXXX:file-system/fs-XXXXXX

plugins:
  - serverless-iam-roles-per-function

package:
  individually: true
  exclude:
    - '**/*'
  include:
    - handler.py

handler.py：

import json
import boto3

s3 = boto3.client('s3', region_name = 'us-west-2')

def download_files_to_efs(event, context):
        """
        Locates the S3 file name (i.e. S3 object "key" value) the initiated the Lambda call, then downloads the file
        into the locally attached EFS drive at the target location.
        :param: event | S3 event record
        :return: dict
        """
        print(event)

        key = event.get('Records')[0].get('s3').get('object').get('key') # bucket: trigger-test, key: uploads/test.vcf
        efs_loci = f"/mnt/efs/vcfs/{key.split('/')[-1]}" # '/mnt/efs/vcfs/test.vcf
        print("key: %s, efs_loci: %s" % (key, efs_loci))
        result = s3.download_file('trigger-test', key, efs_loci)
        if result:
            print('Download Success...')
        else:
            print('Download failed...')
        return { 'status_code': 200 }

EFS 接入点详细信息：

• 详情
  • Root directory path: /vcfs
• POSIX
  • USER ID: 1000
  • Group ID: 1000
• 根目录创建权限
  • Owner User ID: 1000
  • Owner Group ID: 1000
  • POSIX permissions to apply to the root directory path: 777

Answer 1

您的本地路径是localMountPath: /mnt/efs。所以在你的代码中你应该只使用这个路径（不是/mnt/efs/vcfs）：

efs_loci = f"/mnt/efs/{key.split('/')[-1]}" # '/mnt/efs/test.vcf