使用 Python 将文件移入和移出 Amazon S3 存储桶键

Question

我无权访问根存储桶，但我可以访问存储桶中的密钥（KEY NAME）。

示例：我无法访问“BUCKET NAME”，但可以访问“BUCKET NAME/KEY NAME”

我一直在尝试在“KEY NAME”中移动文件。在下面的代码中，我设法开始工作的是 list_objects_v2。

upload_file 给我以下错误：

调用PutObject操作时发生错误（AccessDenied）：访问被拒绝

download_file 给我以下错误：

PermissionError: [WinError 5] 访问被拒绝：'C/Users/username/Desktop'

我对 AWS 环境非常陌生。我该怎么做才能完全获得所需的访问权限？

import logging
import sys
import boto3
import boto
import boto.s3.connection
from botocore.exceptions import ClientError
from boto3.session import Session


def main():

    arguments = len(sys.argv) - 1

    if arguments < 1:
        print("You must supply a folder name")
        return

    bucket_name = 'BUCKET NAME'
    key_name = 'KEY NAME'
    folder = sys.argv[1]


    s3 = boto3.client('s3')
    objects = s3.list_objects_v2(Bucket = bucket_name,
                                 Prefix = key_name + '/' + folder + '/',
                                 Delimiter = '/')
    i = 1

    #
    # Print the bucket's objects within 'KEY NAME'
    #
    if objects is not None:
        # List the object names
        logging.info('Objects in {bucket_name}')
        print("Length of Objects: " + str(len(objects)))
        for obj in objects:
            print("......\n")
            print(i)
            print("....\n")
            print(obj)
            print("..\n")
            print(objects[obj])
            i += 1
    else:
        # Didn't get any keys
        logging.info('No objects in {bucket_name}')

    #
    # Test to see if we can isolate a folder within 'KEY NAME'
    #
    print("\n")
    print("Common Prefixes" + str(objects['CommonPrefixes']) + "\n")
    keys = objects['CommonPrefixes']
    print ("Object 0" + str(keys[0]) + '\n')

    s3 = boto3.resource('s3')
    s3.meta.client.upload_file('C:/Users/username/Desktop/Test/Test.txt',
                               bucket_name,
                               key_name)
    # s3.meta.client.download_file(bucket_name,
    #                              key_name + '/' + folder + '/' + 'Test.txt',
    #                              'C:/Users/username/Desktop')

if __name__ == '__main__':
    main()

Answer 1

最重要的部分是确保您已获得足够的权限来上传/下载/列出前缀。

这是一个示例策略，它授予对 special/ 前缀的访问权限：

{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Sid": "AllowUserToSeeBucketListInTheConsole",
            "Action": [
                "s3:ListAllMyBuckets",
                "s3:GetBucketLocation"
            ],
            "Effect": "Allow",
            "Resource": [
                "arn:aws:s3:::*"
            ]
        },
        {
            "Sid": "AllowListingOfPrefix",
            "Action": [
                "s3:ListBucket"
            ],
            "Effect": "Allow",
            "Resource": [
                "arn:aws:s3:::my-bucket"
            ],
            "Condition": {
                "StringEquals": {
                    "s3:prefix": [
                        "special/"
                    ],
                    "s3:delimiter": [
                        "/"
                    ]
                }
            }
        },
        {
            "Sid": "UploadDownload",
            "Effect": "Allow",
            "Action": [
                "s3:PutObject",
                "s3:GetObject"
            ],
            "Resource": "arn:aws:s3:::my-bucket/special/*"
        }
    ]
}

然后，你可以像这样运行代码：

import boto3

s3_client = boto3.client('s3')

# Upload a file to S3
s3_client.upload_file('/tmp/hello.txt', 'my-bucket', 'special/hello.txt')

# Download an object
s3_client.download_file('my-bucket', 'special/hello.txt', '/tmp/hello2.txt')

# List objects using Client method
response = s3_client.list_objects_v2(Bucket='my-bucket',Delimiter='/',Prefix='special/')
for object in response['Contents']:
  print(object['Key'], object['Size'])

# List objects using Resource method
s3_resource = boto3.resource('s3')
bucket = s3_resource.Bucket('my-bucket')

for object in bucket.objects.filter(Delimiter='/',Prefix='special/'):
  print(object.key, object.size)