IntelliJ：为 gradle 运行配置设置 AWS 环境变量

Question

我正在尝试为配置了 gradle 的 Java Spring Boot 设置我的本地开发环境，它需要一个工作的 AWS CLI 环境。请务必注意，存在不同的配置文件 (.aws/config)（default 未配置），它们是具有 MFA (mfa_serial) 的不同角色切换 (role_arn)。

---

说明

目前我使用的是 Windows 10 Build 18363（“19.09”）、IntelliJ Ultimate 20.02、gradle 5.6.4。 为了运行这个应用程序，我使用 gradle clean bootRun 任务和以下环境变量设置了运行配置：

• AWS_SECRET_ACCESS_KEY
• AWS_ACCESS_KEY_ID
• AWS_REGION
• AWS_DEFAULT_REGION
• AWS_PROFILE
• AWS_DEFAULT_PROFILE

#.aws/config
[profile prod]
region=eu-central-1
output=json
role_arn=<role_arn_prod>
source_profile=site-iam
mfa_serial=<arn_iam_user_mfa_1>


[profile sit]
region=eu-central-1
output=json
role_arn=<role_arn_sit>
source_profile=site-iam
mfa_serial=<arn_iam_user_mfa_1>

[profile dev]
region=eu-central-1
output=json
role_arn=<role_arn_dev>
source_profile=site-iam
mfa_serial=<arn_iam_user_mfa_1>

[profile site-iam]
region=eu-central-1
output=json

[default]

#.aws/credentials
[default]
aws_access_key_id = <access_key_id_1>
aws_secret_access_key = <secret_access_key_1>

[site-iam]
aws_access_key_id = <access_key_id_1>
aws_secret_access_key = <secret_access_key_1>

---

行为

这个java应用程序内部的请求非常简单。

GetObjectRequest getObjectRequest = GetObjectRequest.builder()
    .bucket(this.bucket)
    .key(this.key)
    .build();
String response = this
    .s3Client
    .getObjectAsBytes(getObjectRequest).asString(Charset.forName(StandardCharsets.UTF_8.name()));

以下错误表明没有角色切换和/或身份验证失败（因为我们需要提交 TOTP 才能进行身份验证。

Caused by: software.amazon.awssdk.services.s3.model.S3Exception: Access Denied (Service: S3, Status Code: 403, Request ID: <REQUEST_ID>)
    at software.amazon.awssdk.core.internal.http.CombinedResponseHandler.handleErrorResponse(CombinedResponseHandler.java:123) ~[sdk-core-2.13.13.jar:na]
    at software.amazon.awssdk.core.internal.http.CombinedResponseHandler.handleResponse(CombinedResponseHandler.java:79) ~[sdk-core-2.13.13.jar:na]
    at software.amazon.awssdk.core.internal.http.CombinedResponseHandler.handle(CombinedResponseHandler.java:59) ~[sdk-core-2.13.13.jar:na]
    at software.amazon.awssdk.core.internal.http.CombinedResponseHandler.handle(CombinedResponseHandler.java:40) ~[sdk-core-2.13.13.jar:na]
    at software.amazon.awssdk.core.internal.http.pipeline.stages.HandleResponseStage.execute(HandleResponseStage.java:40) ~[sdk-core-2.13.13.jar:na]
    at software.amazon.awssdk.core.internal.http.pipeline.stages.HandleResponseStage.execute(HandleResponseStage.java:30) ~[sdk-core-2.13.13.jar:na]
    at software.amazon.awssdk.core.internal.http.pipeline.RequestPipelineBuilder$ComposingRequestPipelineStage.execute(RequestPipelineBuilder.java:206) ~[sdk-core-2.13.13.jar:na]
    at software.amazon.awssdk.core.internal.http.pipeline.stages.ApiCallAttemptTimeoutTrackingStage.execute(ApiCallAttemptTimeoutTrackingStage.java:73) ~[sdk-core-2.13.13.jar:na]
    at software.amazon.awssdk.core.internal.http.pipeline.stages.ApiCallAttemptTimeoutTrackingStage.execute(ApiCallAttemptTimeoutTrackingStage.java:42) ~[sdk-core-2.13.13.jar:na]
    at software.amazon.awssdk.core.internal.http.pipeline.stages.TimeoutExceptionHandlingStage.execute(TimeoutExceptionHandlingStage.java:77) ~[sdk-core-2.13.13.jar:na]
    at software.amazon.awssdk.core.internal.http.pipeline.stages.TimeoutExceptionHandlingStage.execute(TimeoutExceptionHandlingStage.java:39) ~[sdk-core-2.13.13.jar:na]
    at software.amazon.awssdk.core.internal.http.pipeline.stages.RetryableStage.execute(RetryableStage.java:64) ~[sdk-core-2.13.13.jar:na]
    at software.amazon.awssdk.core.internal.http.pipeline.stages.RetryableStage.execute(RetryableStage.java:34) ~[sdk-core-2.13.13.jar:na]
    at software.amazon.awssdk.core.internal.http.pipeline.RequestPipelineBuilder$ComposingRequestPipelineStage.execute(RequestPipelineBuilder.java:206) ~[sdk-core-2.13.13.jar:na]
    at software.amazon.awssdk.core.internal.http.StreamManagingStage.execute(StreamManagingStage.java:56) ~[sdk-core-2.13.13.jar:na]
    at software.amazon.awssdk.core.internal.http.StreamManagingStage.execute(StreamManagingStage.java:36) ~[sdk-core-2.13.13.jar:na]
    at software.amazon.awssdk.core.internal.http.pipeline.stages.ApiCallTimeoutTrackingStage.executeWithTimer(ApiCallTimeoutTrackingStage.java:80) ~[sdk-core-2.13.13.jar:na]
    at software.amazon.awssdk.core.internal.http.pipeline.stages.ApiCallTimeoutTrackingStage.execute(ApiCallTimeoutTrackingStage.java:60) ~[sdk-core-2.13.13.jar:na]
    at software.amazon.awssdk.core.internal.http.pipeline.stages.ApiCallTimeoutTrackingStage.execute(ApiCallTimeoutTrackingStage.java:42) ~[sdk-core-2.13.13.jar:na]
    at software.amazon.awssdk.core.internal.http.pipeline.RequestPipelineBuilder$ComposingRequestPipelineStage.execute(RequestPipelineBuilder.java:206) ~[sdk-core-2.13.13.jar:na]
    at software.amazon.awssdk.core.internal.http.pipeline.RequestPipelineBuilder$ComposingRequestPipelineStage.execute(RequestPipelineBuilder.java:206) ~[sdk-core-2.13.13.jar:na]
    at software.amazon.awssdk.core.internal.http.pipeline.stages.ExecutionFailureExceptionReportingStage.execute(ExecutionFailureExceptionReportingStage.java:37) ~[sdk-core-2.13.13.jar:na]
    at software.amazon.awssdk.core.internal.http.pipeline.stages.ExecutionFailureExceptionReportingStage.execute(ExecutionFailureExceptionReportingStage.java:26) ~[sdk-core-2.13.13.jar:na]
    at software.amazon.awssdk.core.internal.http.AmazonSyncHttpClient$RequestExecutionBuilderImpl.execute(AmazonSyncHttpClient.java:189) ~[sdk-core-2.13.13.jar:na]
    at software.amazon.awssdk.core.internal.handler.BaseSyncClientHandler.invoke(BaseSyncClientHandler.java:121) ~[sdk-core-2.13.13.jar:na]
    at software.amazon.awssdk.core.internal.handler.BaseSyncClientHandler.doExecute(BaseSyncClientHandler.java:147) ~[sdk-core-2.13.13.jar:na]
    at software.amazon.awssdk.core.internal.handler.BaseSyncClientHandler.execute(BaseSyncClientHandler.java:76) ~[sdk-core-2.13.13.jar:na]
    at software.amazon.awssdk.core.client.handler.SdkSyncClientHandler.execute(SdkSyncClientHandler.java:52) ~[sdk-core-2.13.13.jar:na]
    at software.amazon.awssdk.awscore.client.handler.AwsSyncClientHandler.execute(AwsSyncClientHandler.java:62) ~[aws-core-2.13.13.jar:na]
    at software.amazon.awssdk.services.s3.DefaultS3Client.getObject(DefaultS3Client.java:3606) ~[s3-2.13.13.jar:na]
    at software.amazon.awssdk.services.s3.S3Client.getObjectAsBytes(S3Client.java:7563) ~[s3-2.13.13.jar:na]

类似的行为可以通过 cmd 重现

aws s3 ls

An error occurred (AccessDenied) when calling the ListBuckets operation: Access Denied

---

不同的方法

STS 获取会话令牌 |环境变量

我已经通过

获得了会话令牌

aws sts get-session-token --serial <arn_iam_user_mfa_1> --token-code <TOTP> --duration-seconds 129600

作为 json

{
    "Credentials": {
        "AccessKeyId": "<access_key_id_2>",
        "SecretAccessKey": "<secret_access_key_2>",
        "SessionToken": "<session_token_2>",
        "Expiration": "2020-08-02T22:57:12+00:00"
    }
}

并将以下环境变量设置为获得的值：

• AWS_ACCESS_KEY_ID: <access_key_id_2>
• AWS_SECRET_ACCESS_KEY: <secret_access_key_2>
• AWS_SESSION_TOKEN: <session_token_2>
• AWS_SECURITY_TOKEN: <session_token_2>

但这会返回相同的错误

Caused by: software.amazon.awssdk.services.s3.model.S3Exception: Access Denied (Service: S3, Status Code: 403, Request ID: <request_id>)

---

又快又脏.aws/config修复

经过多次调试，default 配置文件更改如下

[default]
region=eu-central-1
output=json
role_arn=<role_arn_prod>
source_profile=site-iam
mfa_serial=<arn_iam_user_mfa_1>

导致

Caused by: software.amazon.awssdk.core.exception.SdkClientException: Unable to load credentials from any of the providers in the chain AwsCredentialsProviderChain(credentialsProviders=[SystemPropertyCredentialsProvider(), EnvironmentVariableCredentialsProvider(), WebIdentityTokenCredentialsProvider(), ProfileCredentialsProvider(), ContainerCredentialsProvider(), InstanceProfileCredentialsProvider()]) : [SystemPropertyCredentialsProvider(): Unable to load credentials from system settings. Access key must be specified either via environment variable (AWS_ACCESS_KEY_ID) or system property (aws.accessKeyId)., EnvironmentVariableCredentialsProvider(): Unable to load credentials from system settings. Access key must be specified either via environment variable (AWS_ACCESS_KEY_ID) or system property (aws.accessKeyId)., WebIdentityTokenCredentialsProvider(): Either the environment variable AWS_WEB_IDENTITY_TOKEN_FILE or the javaproperty aws.webIdentityTokenFile must be set., ProfileCredentialsProvider(): To use assumed roles in the '<ROLE_PROFILE>' profile, the 'sts' service module must be on the class path., ContainerCredentialsProvider(): Cannot fetch credentials from container - neither AWS_CONTAINER_CREDENTIALS_FULL_URI or AWS_CONTAINER_CREDENTIALS_RELATIVE_URI environment variables are set., InstanceProfileCredentialsProvider(): Unable to load credentials from service endpoint.]

---

设置AWS_WEB_IDENTITY_TOKEN_FILE

AWS_WEB_IDENTITY_TOKEN_FILE env var 已通过 WSL 样式路径 (\wsl$\...) 设置为位于 ~/.aws/cli/cache 中的现有令牌文件 即使环境变量存在和/或更新，也会导致与上述相同的错误

Caused by: software.amazon.awssdk.core.exception.SdkClientException: Unable to load credentials from any of the providers in the chain AwsCredentialsProviderChain(credentialsProviders=[SystemPropertyCredentialsProvider(), EnvironmentVariableCredentialsProvider(), WebIdentityTokenCredentialsProvider(), ProfileCredentialsProvider(), ContainerCredentialsProvider(), InstanceProfileCredentialsProvider()]) : [SystemPropertyCredentialsProvider(): Unable to load credentials from system settings. Access key must be specified either via environment variable (AWS_ACCESS_KEY_ID) or system property (aws.accessKeyId)., EnvironmentVariableCredentialsProvider(): Unable to load credentials from system settings. Access key must be specified either via environment variable (AWS_ACCESS_KEY_ID) or system property (aws.accessKeyId)., WebIdentityTokenCredentialsProvider(): Either the environment variable AWS_WEB_IDENTITY_TOKEN_FILE or the javaproperty aws.webIdentityTokenFile must be set., ProfileCredentialsProvider(): To use assumed roles in the '<ROLE_PROFILE>' profile, the 'sts' service module must be on the class path., ContainerCredentialsProvider(): Cannot fetch credentials from container - neither AWS_CONTAINER_CREDENTIALS_FULL_URI or AWS_CONTAINER_CREDENTIALS_RELATIVE_URI environment variables are set., InstanceProfileCredentialsProvider(): Unable to load credentials from service endpoint.]

---

调试

没有权限问题。 此特定文件的下载适用于正确的角色 arn。

C:\Users\tunnelblick>aws s3 ls

An error occurred (AccessDenied) when calling the ListBuckets operation: Access Denied

C:\Users\tunnelblick>aws s3 ls --profile prod
Enter MFA code for <arn_iam_user_mfa_1>:
2020-02-05 09:28:41 <bucket_1>
2020-06-23 05:16:07 <bucket_2>

C:\Users\tunnelblick>aws s3 cp s3://<bucket_1>/<file>.json .
fatal error: An error occurred (403) when calling the HeadObject operation: Forbidden

C:\Users\tunnelblick>aws s3 cp s3://<bucket_1>/<file>.json . --profile prod
download: s3://<bucket_1>/<file>.json to .\<file>.json

在我的 WSL 1 终端中设置这些环境变量并执行 aws s3 ls 导致相同的 Access Denied 错误 (An error occurred (AccessDenied) when calling the ListBuckets operation: Access Denied)

Answer 1

后来我也尝试在~/.aws/credentials 文件中设置这些值。似乎我错过了一些可以用aws-mfa（AWS STS 的 python 包装器）解决的选项。 我正在使用以下example

aws-mfa --duration 1800 --device arn:aws:iam::123456788990:mfa/dudeman --assume-role arn:aws:iam::123456788990:role/some-role --role-session-name some-role-session`

这会在default 配置文件中填充其他选项，以进行正确的角色切换，例如assumed_role、assume_role_arn、aws_session_token、aws_security_token、expiration，并且与我的运行配置一样工作。