AWS 托管 elasticsearch S3 快照权限答案

【问题标题】：AWS managed elasticsearch S3 snapshot permissionsAWS 托管 elasticsearch S3 快照权限
【发布时间】：2021-10-25 17:13:50
【问题描述】：

我有一个 AWS 托管的 Elasticsearch 实例（较旧的“OpenSearch”变体）。我想启用到 S3 的快照备份，但我似乎无法获得正确的权限。

我正在尝试使用实例角色，而不是特定的用户凭据。我试过关注this guide 以及其他一些非常相似的人。

我有一个角色：arn:aws:iam::MYACCOUNTID:role/QA-elasticsearch-instance-role。该角色具有以下政策：

{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Sid": "AllowListS3BackupBucket",
            "Action": [
                "iam:PassRole",
                "s3:ListBucket",
                "s3:GetBucketLocation",
                "s3:ListBucketMultipartUploads",
                "s3:ListBucketVersions"
            ],
            "Effect": "Allow",
            "Resource": [
                "arn:aws:iam::MYACCOUNTID:role/QA-elasticsearch-instance-role",
                "arn:aws:s3:::qa-mycompanyname-elasticsearch-backups"
            ]
        },
        {
            "Sid": "AllowBackupStorage",
            "Action": [
                "s3:GetObject",
                "s3:PutObject",
                "s3:DeleteObject",
                "s3:AbortMultipartUpload",
                "s3:ListMultipartUploadParts"
            ],
            "Effect": "Allow",
            "Resource": [
                "arn:aws:s3:::qa-mycompanyname-elasticsearch-backups/*"
            ]
        },
        {
            "Sid": "AllowElasticSearchPost",
            "Effect": "Allow",
            "Action": "es:ESHttpPost",
            "Resource": "arn:aws:es:us-east-1:MYACCOUNTID:domain/qa-elasticsearch"
        }
    ]
}

此角色还具有以下信任关系：

{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Sid": "",
      "Effect": "Allow",
      "Principal": {
        "Service": "es.amazonaws.com"
      },
      "Action": "sts:AssumeRole"
    }
  ]
}

我有以下 python 脚本来尝试创建快照存储库：

import boto3
import requests
from requests_aws4auth import AWS4Auth

host = 'https://MYELASTICSEARCHENDPOINT.us-east-1.es.amazonaws.com/'
region = 'us-east-1' # For example, us-west-1
service = 'es'
credentials = boto3.Session().get_credentials()

awsauth = AWS4Auth("MYACCESSKEY", "MYSECRETKEY", region, service)

# Register repository
path = '_snapshot/qa_snapshot_repository' # the Elasticsearch API endpoint
url = host + path

payload = {
    "type": "s3",
    "settings": {
        "bucket": "qa-MYCOMPANYNAME-elasticsearch-backups",
        "region": "us-east-1",
        "role_arn": "arn:aws:iam::MYACCOUNTID:role/QA-elasticsearch-instance-role"
    }
}

headers = {"Content-Type": "application/json"}

r = requests.put(url, auth=awsauth, json=payload, headers=headers)

print(r.status_code)
print(r.text)

当我运行它时，(python3 elasticsearch_test.py | jq .) 我得到以下信息：

{
  "error": {
    "root_cause": [
      {
        "type": "repository_verification_exception",
        "reason": "[qa_snapshot_repository] path  is not accessible on master node"
      }
    ],
    "type": "repository_verification_exception",
    "reason": "[qa_snapshot_repository] path  is not accessible on master node",
    "caused_by": {
      "type": "i_o_exception",
      "reason": "Unable to upload object [tests-4-VcUFXhTcW4PQ6fi6AiVA/master.dat] using a single upload",
      "caused_by": {
        "type": "amazon_s3_exception",
        "reason": "Access Denied (Service: Amazon S3; Status Code: 403; Error Code: AccessDenied; Request ID: RJKPNBBEQQBNWHWT; S3 Extended Request ID: 3O5AqSkKhsCNlmYLCtcfnE0omhp6wdbPCejJ2r4dNJq93l2xppSs4Hv6uTGvNIJ2jFWLt31gsyc=)"
      }
    }
  },
  "status": 500
}

我已经在谷歌上搜索并尝试调整权限几个小时了，但似乎无法做到正确。我有什么明显的遗漏吗？

【问题讨论】：

Amazon OpenSearch Service documentation 显示iam:PassRole 权限已授予用于签署请求的 IAM 用户或角色。此角色与 Amazon OpenSearch Service 为获得对 S3 存储桶的访问权限而代入的角色不同。
@ChinHuang - 好的，谢谢。对不起，我很抱歉，但我该怎么处理这些信息？我已经为 elasticsearch 域添加了信任关系来承担该角色，并且该角色具有读取/写入 S3 的所有相关权限。我将该角色传递给快照存储库创建请求。我做错了什么？

标签： elasticsearch amazon-s3 aws-elasticsearch

【解决方案1】：

所以事实证明错误响应是超级误导。这并不是真正的“AccessDenied”问题，而是由于客户端和服务器之间的加密不正确，对 S3 的请求被拒绝。在this post 之后，我决定尝试添加"server_side_encryption": "true"，一切正常。

payload = {
    "type": "s3",
    "settings": {
        "bucket": "qa-mycompanyname-elasticsearch-backups",
        "region": "us-east-1",
        "role_arn": "arn:aws:iam::myaccountid:role/QA-elasticsearch-instance-role",
        "server_side_encryption": "true"
    }
}

200
{
  "acknowledged": true
}

我没有正确解释为什么会这样。我猜它告诉 Elasticsearch 实例上的客户端使用 https URL 与 S3 通信而不是 http，但似乎很疯狂，http 将是默认设置，或者不会有更好的错误消息返回。

【讨论】：