使用 scala 时 lambda 的 AWS 凭证不起作用答案

【问题标题】：AWS Credentials for lambda when working with scala not working使用 scala 时 lambda 的 AWS 凭证不起作用
【发布时间】：2018-05-25 14:30:30
【问题描述】：

AWS lambda 函数在尝试使用使用 DefaultCredentialProvider 提供的凭证时不起作用。

我需要将凭据传递给 S3 才能运行。

代码

def initializeAwsCredentials():AWSCredentials = {
    var credentials: AWSCredentials  = null
    try {
      credentials = new ProfileCredentialsProvider().getCredentials
    } catch {
      case e: Exception => {
        throw new AmazonClientException(
          "Cannot load the credentials from the credential profiles file. " +
            "Please make sure that your credentials file is at the correct " +
            "location (~/.aws/credentials), and is in valid format.",
          e);
      }

    }
    return credentials
  }

 def buildS3API(credentials: AWSCredentials): AmazonS3 = {
  new AmazonS3Client(credentials)
}

// inside handle request
val credentials = initializeAwsCredentials()
println("Credetials have been retrieved successfully")

println("Build S3 API using the constructor provided")
val s3 = buildS3API(credentials)
s3.setRegion(region)
println("S3 API is now available")

错误

{
  "errorMessage": "Cannot load the credentials from the credential profiles file. Please make sure that your credentials file is at the correct location (~/.aws/credentials), and is in valid format.",
  "errorType": "com.amazonaws.AmazonClientException",
  "stackTrace": [
    "example.Main$.initializeAwsCredentials(Hello.scala:52)",
    "example.Main$.handleRequest(Hello.scala:125)",
    "example.Main.handleRequest(Hello.scala)",
    "sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)",
    "sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)",
    "sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)",
    "java.lang.reflect.Method.invoke(Method.java:498)"
  ],
  "cause": {
    "errorMessage": "java.lang.NullPointerException",
    "errorType": "java.lang.NullPointerException",
    "stackTrace": [
      "com.amazonaws.auth.profile.ProfilesConfigFile.<init>(ProfilesConfigFile.java:143)",
      "com.amazonaws.auth.profile.ProfilesConfigFile.<init>(ProfilesConfigFile.java:132)",
      "com.amazonaws.auth.profile.ProfilesConfigFile.<init>(ProfilesConfigFile.java:99)",
      "com.amazonaws.auth.profile.ProfileCredentialsProvider.getCredentials(ProfileCredentialsProvider.java:135)",
      "example.Main$.initializeAwsCredentials(Hello.scala:45)",
      "example.Main$.handleRequest(Hello.scala:125)",
      "example.Main.handleRequest(Hello.scala)",
      "sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)",
      "sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)",
      "sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)",
      "java.lang.reflect.Method.invoke(Method.java:498)"
    ]
  }
}

更新

使用 InstanceProfileCredentialsProvider 会引发错误：

val provider: InstanceProfileCredentialsProvider = new InstanceProfileCredentialsProvider()
credentials = provider.getCredentials()

给我错误：

"cause": {
    "errorMessage": "Unable to load credentials from Amazon EC2 metadata service",
    "errorType": "com.amazonaws.AmazonClientException",
    "stackTrace": [
      "com.amazonaws.auth.InstanceProfileCredentialsProvider.handleError(InstanceProfileCredentialsProvider.java:244)",
      "com.amazonaws.auth.InstanceProfileCredentialsProvider.loadCredentials(InstanceProfileCredentialsProvider.java:225)",
      "com.amazonaws.auth.InstanceProfileCredentialsProvider.getCredentials(InstanceProfileCredentialsProvider.java:124)",
      "example.Main$.initializeAwsCredentials(Hello.scala:46)",
      "example.Main$.handleRequest(Hello.scala:126)",
      "example.Main.handleRequest(Hello.scala)",
      "sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)",
      "sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)",
      "sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)",
      "java.lang.reflect.Method.invoke(Method.java:498)"
    ],
    "cause": {
      "errorMessage": "Connection refused (Connection refused)",
      "errorType": "java.net.ConnectException",
      "stackTrace": [
        "java.net.PlainSocketImpl.socketConnect(Native Method)",
        "java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:350)",
        "java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:206)",
        "java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:188)",
        "java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392)",
        "java.net.Socket.connect(Socket.java:589)",
        "sun.net.NetworkClient.doConnect(NetworkClient.java:175)",
        "sun.net.www.http.HttpClient.openServer(HttpClient.java:463)",
        "sun.net.www.http.HttpClient.openServer(HttpClient.java:558)",
        "sun.net.www.http.HttpClient.<init>(HttpClient.java:242)",
        "sun.net.www.http.HttpClient.New(HttpClient.java:339)",
        "sun.net.www.http.HttpClient.New(HttpClient.java:357)",
        "sun.net.www.protocol.http.HttpURLConnection.getNewHttpClient(HttpURLConnection.java:1202)",
        "sun.net.www.protocol.http.HttpURLConnection.plainConnect0(HttpURLConnection.java:1138)",
        "sun.net.www.protocol.http.HttpURLConnection.plainConnect(HttpURLConnection.java:1032)",
        "sun.net.www.protocol.http.HttpURLConnection.connect(HttpURLConnection.java:966)",
        "com.amazonaws.internal.EC2MetadataClient.readResource(EC2MetadataClient.java:90)",
        "com.amazonaws.internal.EC2MetadataClient.getDefaultCredentials(EC2MetadataClient.java:55)",
        "com.amazonaws.auth.InstanceProfileCredentialsProvider.loadCredentials(InstanceProfileCredentialsProvider.java:186)",
        "com.amazonaws.auth.InstanceProfileCredentialsProvider.getCredentials(InstanceProfileCredentialsProvider.java:124)",
        "example.Main$.initializeAwsCredentials(Hello.scala:46)",
        "example.Main$.handleRequest(Hello.scala:126)",
        "example.Main.handleRequest(Hello.scala)",
        "sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)",
        "sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)",
        "sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)",
        "java.lang.reflect.Method.invoke(Method.java:498)"
      ]
    }
  }
}

在使用 lambda 时将以下配置为环境变量也会失败：

Lambda was unable to configure your environment variables because the 
environment variables you have provided contains reserved keys that are 
currently not supported for modification. Reserved keys used in this 
request: AWS_ACCESS_KEY_ID, AWS_SECRET_ACCESS_KEY

【问题讨论】：

标签： scala amazon-web-services amazon-s3 aws-lambda aws-sdk

【解决方案1】：

对于 Lambda 函数，您需要使用 IAM 角色作为凭证。然后，您可以使用 DefaultAWSCredentialsProviderChain 或 InstanceProfileCredentialsProvider 从 IAM 角色检索凭证。

Class InstanceProfileCredentialsProvider

这是一个使用 InstanceProfileCredentialsProvider 的示例：

   AWSCredentialsProvider credentialsProvider = null;
    try {
        credentialsProvider = new InstanceProfileCredentialsProvider();
        // Verify we can fetch credentials
        credentialsProvider.getCredentials();
        System.out.println("Obtained credentials.");
    } catch (AmazonClientException e) {
        System.out.println("Unable to obtain credentials", e);
        return -1;
    }

    System.out.println("Using credentials with access key id: " + credentialsProvider.getCredentials().getAWSAccessKeyId());

【讨论】：

你能展示一下吗？例如，我使用的版本似乎没有推荐的电阻 mwthod
添加语句 import com.amazonaws.auth.InstanceProfileCredentialsProvider 或仅 import com.amazonaws.auth.*
我收到一个错误Unable to load credentials from Amazon EC2 metadata service，代码为val provider: InstanceProfileCredentialsProvider = new InstanceProfileCredentialsProvider() credentials = provider.getCredentials()
我添加了一些代码来展示如何获取您的凭据。 Q) 您是否为您的 Lambda 函数分配了 IAM 角色？
@atom88 - 好提示。当我在 2017 年回答这个问题时，V2 SDK 还不存在。 V2 于 2018 年 11 月发布。有时我们必须维护遗留代码，所以我留下了答案。

【解决方案2】：

我不确定您是否需要明确的凭据提供程序。在 AWS Lambda 中，凭证是通过 lambda 可以承担的角色自动提供的。我知道我从来没有明确地这样做过。

http://docs.aws.amazon.com/sdk-for-javascript/v2/developer-guide/loading-node-credentials-lambda.html

【讨论】：

【解决方案3】：

我在java中有类似的情况。希望同样的方法在这里适用。我将本地切换到 EnvironmentVariableCredentialsProvider，通过本地 java 程序的“运行配置”中的 env 选项卡提供凭据。然后我将 jar 上传到 Lambda 函数，也有 Env 配置，只需在那里提供凭据。

【讨论】：

【解决方案4】：

在适用于 Java 的 AWS 开发工具包 (1.11.4xx) 的最新版本中，几乎所有服务都具有可用于快速创建客户端的“客户端构建器”。

val snsClient = AmazonSNSClientBuilder.defaultClient()
val s3Client = AmazonS3ClientBuilder.defaultClient()
val dynamoDbClient = AmazonDynamoDBAsyncClientBuilder.defaultClient()
val sesClient = AmazonSimpleEmailServiceAsyncClientBuilder.defaultClient()
// ...

在 Lambda 中，defaultClient() 工作得非常好，因为它将创建一个使用适当提供程序的客户端。此提供程序使用具有 lambda 执行角色中定义的权限的凭据。

在本地环境中，defaultClient 也可以正常工作，因为它会获取主机凭据。这是因为 defaultClient 使用在寻找

的 default credentials provider chain

环境变量
Java 系统属性
凭据配置文件
ECS 容器凭据
实例配置文件凭据

此方法也很简洁，但您也可以使用客户端构建器使用特定凭据“设置/配置”创建客户端。

AWS SDK for Java v2

如果您想使用新版本的 Java SDK (>=2.1)，可以使用 create 方法来获取客户端（虽然我只用它来试验新的 SDK）

val s3Client = S3AsyncClient.create()
val dynamoDbClient = DynamoDbAsyncClient.create()
// ...

【讨论】：