【问题标题】：Can't get S3 bucket names based on tag values无法根据标签值获取 S3 存储桶名称
【发布时间】：2019-08-12 10:14:40
【问题描述】：

在我们的 aws 账户中，我们有大约 1000 个 S3 存储桶，每个 S3 存储桶都有对应于应用程序名称的标签（例如，key=application，value=app1）。我试图找出特定应用程序拥有多少个 S3 存储桶。因此，首先得到所有 S3 存储桶的列表；然后遍历列表以匹配“app1”的标记值。它应该很简单，但由于某种原因，它给出了“调用 GetBucketTagging 操作时拒绝访问”错误。我验证了我假设的 IAM 角色具有 GetBucketTagging 权限

1) 使用凭证获取 s3 存储桶列表（我假设为 IAM 角色） 2）遍历列表并尝试匹配标签的键值对（key=application，value=application1）

第一个选项

import boto3
client = boto3.client('s3')
buckets = client.list_buckets()['Buckets']
matching_buckets = []
# tag key and value to search for
tag_key = 'application'
tag_value = 'app1'
for bucket in buckets:
    tags = client.get_bucket_tagging(Bucket=bucket['Name'])['TagSet']

    for tag in tags:
        if tag['Key'] == tag_key and tag['Value'] == tag_value:
        matching_buckets.append(bucket['Name'])

第二个选项

import boto3
s3 = boto3.client('s3')
app = "app1"
bucketlist = s3.list_buckets()['Buckets']
print(len(bucketlist))
bucketname = []
n=0
#iterate thru the list of {Name, CreationDate} to get all the bucket names and append to empty list

def bucket_tagging_method(b,app):
    mybucketlist = []
    bucket_tagging = s3.get_bucket_tagging(Bucket=b)
    tag_set = bucket_tagging['TagSet']
    for tag in tag_set:
        if (tag['Key'] == "application") and (tag['Value'] == app) :
            mybucketlist.append(b)
            pass
    return(mybucketlist)


while n < len(bucketlist):
    d = bucketlist[n]
    bucketname.append(d['Name'])
    n+=1

for i in bucketname:
    print(bucket_tagging_method(i,app))

它给出了以下错误

tags = client.get_bucket_tagging(Bucket=bucket['Name'])['TagSet']
  File "/usr/local/lib/python3.7/site-packages/botocore/client.py", line 357, in _api_call
    return self._make_api_call(operation_name, kwargs)
  File "/usr/local/lib/python3.7/site-packages/botocore/client.py", line 661, in _make_api_call
    raise error_class(parsed_response, operation_name)
botocore.exceptions.ClientError: An error occurred (AccessDenied) when calling the GetBucketTagging operation: Access Denied

【问题讨论】：

在担任角色后如何将凭据传递给您的应用程序？
我运行一个实用程序来验证我的假定角色和配置（在 .aws 文件夹下）文件得到相应更新
如果实用程序创建命名配置文件，您可能需要导出一些环境变量来激活配置文件。首先在 .aws/credentials 中找到此配置文件的名称，然后将其导出，导出 AWS_PROFILE=your_profile_name"，然后再运行您的 python 应用程序。
事实证明，有些桶有非常严格的桶策略；因此 bucket_tagging 抛出异常。请在下面查看我的答案。

标签： python python-3.x amazon-s3

【解决方案1】：

import boto3

boto3.setup_default_session(profile_name='my_profile')
client = boto3.client('s3')
# get list of buckets this iam role can see
buckets = client.list_buckets()['Buckets']

# iterate through list of buckets looking at tags
matching_buckets = []
# tag key and value to search for
tag_key = 'app'
tag_value = 'application1'

for idx, bucket in enumerate(buckets):
#comment out following line if you don't want to see the progress
#print(f'{idx+1} of {len(buckets)} - {bucket["Name"]}')
try:
    tags = client.get_bucket_tagging(Bucket=bucket['Name'])['TagSet']
except client.exceptions.ClientError:
    continue
# iterate through tags looking for specific key
for tag in tags:
    if tag['Key'] == tag_key and tag['Value'] == tag_value:
        matching_buckets.append(bucket['Name'])

print("buckets belonging to", tag_value, "are: ", matching_buckets)

【讨论】：