为什么 aws-sdk-ruby 在初始化时连接到“169.254.169.254”？答案

【问题标题】：Why is aws-sdk-ruby connecting to '169.254.169.254' on initialization?为什么 aws-sdk-ruby 在初始化时连接到“169.254.169.254”？
【发布时间】：2021-09-08 06:38:52
【问题描述】：

我安装了 gem webmock，它在测试套件期间阻止了外部连接。安装后，运行测试套件时得到了意想不到的结果。

我有点困惑，为什么 aws-sdk 需要在初始化时连接到 169.254.169.254？

rspec spec/models/concerns/posconcern_spec.rb
[Coveralls] Set up the SimpleCov formatter.
[Coveralls] Using SimpleCov's 'rails' settings.

An error occurred while loading ./spec/models/concerns/posconcern_spec.rb.
Failure/Error: require File.expand_path('../../config/environment', __FILE__)

WebMock::NetConnectNotAllowedError:
  Real HTTP connections are disabled. Unregistered request: GET http://169.254.169.254/latest/meta-data/iam/security-credentials/ with headers {'Accept'=>'*/*', 'Accept-Encoding'=>'gzip;q=1.0,deflate;q=0.6,identity;q=0.3', 'User-Agent'=>'Ruby'}

  You can stub this request with the following snippet:

  stub_request(:get, "http://169.254.169.254/latest/meta-data/iam/security-credentials/").
    with(headers: {'Accept'=>'*/*', 'Accept-Encoding'=>'gzip;q=1.0,deflate;q=0.6,identity;q=0.3', 'User-Agent'=>'Ruby'}).
    to_return(status: 200, body: "", headers: {})

  ============================================================
# /home/andey/.rvm/gems/ruby-2.3.4/gems/webmock-3.1.0/lib/webmock/http_lib_adapters/net_http.rb:114:in `request'
# /home/andey/.rvm/gems/ruby-2.3.4/gems/aws-sdk-core-2.10.52/lib/aws-sdk-core/instance_profile_credentials.rb:109:in `http_get'
# /home/andey/.rvm/gems/ruby-2.3.4/gems/aws-sdk-core-2.10.52/lib/aws-sdk-core/instance_profile_credentials.rb:90:in `block (2 levels) in get_credentials'
# /home/andey/.rvm/gems/ruby-2.3.4/gems/aws-sdk-core-2.10.52/lib/aws-sdk-core/instance_profile_credentials.rb:105:in `open_connection'
# /home/andey/.rvm/gems/ruby-2.3.4/gems/aws-sdk-core-2.10.52/lib/aws-sdk-core/instance_profile_credentials.rb:88:in `block in get_credentials'
# /home/andey/.rvm/gems/ruby-2.3.4/gems/aws-sdk-core-2.10.52/lib/aws-sdk-core/instance_profile_credentials.rb:121:in `retry_errors'
# /home/andey/.rvm/gems/ruby-2.3.4/gems/aws-sdk-core-2.10.52/lib/aws-sdk-core/instance_profile_credentials.rb:87:in `get_credentials'
# /home/andey/.rvm/gems/ruby-2.3.4/gems/aws-sdk-core-2.10.52/lib/aws-sdk-core/instance_profile_credentials.rb:73:in `block in refresh'
# /home/andey/.rvm/gems/ruby-2.3.4/gems/aws-sdk-core-2.10.52/lib/aws-sdk-core/instance_profile_credentials.rb:121:in `retry_errors'
# /home/andey/.rvm/gems/ruby-2.3.4/gems/aws-sdk-core-2.10.52/lib/aws-sdk-core/instance_profile_credentials.rb:72:in `refresh'
# /home/andey/.rvm/gems/ruby-2.3.4/gems/aws-sdk-core-2.10.52/lib/aws-sdk-core/refreshing_credentials.rb:20:in `initialize'
# /home/andey/.rvm/gems/ruby-2.3.4/gems/aws-sdk-core-2.10.52/lib/aws-sdk-core/instance_profile_credentials.rb:51:in `initialize'
# /home/andey/.rvm/gems/ruby-2.3.4/gems/aws-sdk-core-2.10.52/lib/aws-sdk-core/credential_provider_chain.rb:90:in `new'
# /home/andey/.rvm/gems/ruby-2.3.4/gems/aws-sdk-core-2.10.52/lib/aws-sdk-core/credential_provider_chain.rb:90:in `instance_profile_credentials'
# /home/andey/.rvm/gems/ruby-2.3.4/gems/aws-sdk-core-2.10.52/lib/aws-sdk-core/credential_provider_chain.rb:12:in `block in resolve'
# /home/andey/.rvm/gems/ruby-2.3.4/gems/aws-sdk-core-2.10.52/lib/aws-sdk-core/credential_provider_chain.rb:11:in `each'
# /home/andey/.rvm/gems/ruby-2.3.4/gems/aws-sdk-core-2.10.52/lib/aws-sdk-core/credential_provider_chain.rb:11:in `resolve'
# /home/andey/.rvm/gems/ruby-2.3.4/gems/aws-sdk-core-2.10.52/lib/aws-sdk-core/plugins/request_signer.rb:37:in `block in <class:RequestSigner>'
# /home/andey/.rvm/gems/ruby-2.3.4/gems/aws-sdk-core-2.10.52/lib/seahorse/client/configuration.rb:70:in `call'
# /home/andey/.rvm/gems/ruby-2.3.4/gems/aws-sdk-core-2.10.52/lib/seahorse/client/configuration.rb:205:in `block in resolve_defaults'
# /home/andey/.rvm/gems/ruby-2.3.4/gems/aws-sdk-core-2.10.52/lib/seahorse/client/configuration.rb:57:in `each'
# /home/andey/.rvm/gems/ruby-2.3.4/gems/aws-sdk-core-2.10.52/lib/seahorse/client/configuration.rb:57:in `each'
# /home/andey/.rvm/gems/ruby-2.3.4/gems/aws-sdk-core-2.10.52/lib/seahorse/client/configuration.rb:204:in `resolve_defaults'
# /home/andey/.rvm/gems/ruby-2.3.4/gems/aws-sdk-core-2.10.52/lib/seahorse/client/configuration.rb:200:in `value_at'
# /home/andey/.rvm/gems/ruby-2.3.4/gems/aws-sdk-core-2.10.52/lib/seahorse/client/configuration.rb:189:in `block in resolve'
# /home/andey/.rvm/gems/ruby-2.3.4/gems/aws-sdk-core-2.10.52/lib/seahorse/client/configuration.rb:189:in `resolve'
# /home/andey/.rvm/gems/ruby-2.3.4/gems/aws-sdk-core-2.10.52/lib/seahorse/client/configuration.rb:177:in `apply_defaults'
# /home/andey/.rvm/gems/ruby-2.3.4/gems/aws-sdk-core-2.10.52/lib/seahorse/client/configuration.rb:150:in `build!'
# /home/andey/.rvm/gems/ruby-2.3.4/gems/aws-sdk-core-2.10.52/lib/seahorse/client/base.rb:68:in `build_config'
# /home/andey/.rvm/gems/ruby-2.3.4/gems/aws-sdk-core-2.10.52/lib/seahorse/client/base.rb:19:in `initialize'
# /home/andey/.rvm/gems/ruby-2.3.4/gems/aws-sdk-core-2.10.52/lib/seahorse/client/base.rb:105:in `new'
# ./config/initializers/aws.rb:1:in `<top (required)>'
# /home/andey/.rvm/gems/ruby-2.3.4/gems/activesupport-4.2.9/lib/active_support/dependencies.rb:268:in `load'
# /home/andey/.rvm/gems/ruby-2.3.4/gems/activesupport-4.2.9/lib/active_support/dependencies.rb:268:in `block in load'
# /home/andey/.rvm/gems/ruby-2.3.4/gems/activesupport-4.2.9/lib/active_support/dependencies.rb:240:in `load_dependency'
# /home/andey/.rvm/gems/ruby-2.3.4/gems/activesupport-4.2.9/lib/active_support/dependencies.rb:268:in `load'
# /home/andey/.rvm/gems/ruby-2.3.4/gems/railties-4.2.9/lib/rails/engine.rb:652:in `block in load_config_initializer'
# /home/andey/.rvm/gems/ruby-2.3.4/gems/activesupport-4.2.9/lib/active_support/notifications.rb:166:in `instrument'
# /home/andey/.rvm/gems/ruby-2.3.4/gems/railties-4.2.9/lib/rails/engine.rb:651:in `load_config_initializer'
# /home/andey/.rvm/gems/ruby-2.3.4/gems/railties-4.2.9/lib/rails/engine.rb:616:in `block (2 levels) in <class:Engine>'
# /home/andey/.rvm/gems/ruby-2.3.4/gems/railties-4.2.9/lib/rails/engine.rb:615:in `each'
# /home/andey/.rvm/gems/ruby-2.3.4/gems/railties-4.2.9/lib/rails/engine.rb:615:in `block in <class:Engine>'
# /home/andey/.rvm/gems/ruby-2.3.4/gems/railties-4.2.9/lib/rails/initializable.rb:30:in `instance_exec'
# /home/andey/.rvm/gems/ruby-2.3.4/gems/railties-4.2.9/lib/rails/initializable.rb:30:in `run'
# /home/andey/.rvm/gems/ruby-2.3.4/gems/railties-4.2.9/lib/rails/initializable.rb:55:in `block in run_initializers'
# /home/andey/.rvm/gems/ruby-2.3.4/gems/railties-4.2.9/lib/rails/initializable.rb:44:in `each'
# /home/andey/.rvm/gems/ruby-2.3.4/gems/railties-4.2.9/lib/rails/initializable.rb:44:in `tsort_each_child'
# /home/andey/.rvm/gems/ruby-2.3.4/gems/railties-4.2.9/lib/rails/initializable.rb:54:in `run_initializers'
# /home/andey/.rvm/gems/ruby-2.3.4/gems/railties-4.2.9/lib/rails/application.rb:352:in `initialize!'
# ./config/environment.rb:5:in `<top (required)>'
# ./spec/rails_helper.rb:3:in `require'
# ./spec/rails_helper.rb:3:in `<top (required)>'
# ./spec/models/concerns/posconcern_spec.rb:1:in `require'
# ./spec/models/concerns/posconcern_spec.rb:1:in `<top (required)>'

【问题讨论】：

标签： ruby amazon-web-services aws-sdk

【解决方案1】：

它正在尝试连接到 AWS 元数据服务器以获取 AWS 凭证。所有 AWS 服务器都应该能够连接到内部元数据服务器。

我安装了 gem webmock，它在测试套件

你是如何阻止它的？防火墙？添加规则以允许流量到169.254.169.254 或存根。

脚本正在尝试连接到 AWS 服务。为此，它需要可以通过多种方式提供的凭据。一种方法是使用 AWS IAM 角色并从元数据服务器 (169.254.169.254) 动态获取凭证。您的脚本正在连接到 169.254.169.254 以获取凭据。以后可以用来连接AWS服务。

【讨论】：

我从不同的角度看待问题。为什么不让它在我的测试套件中运行呢？如何防止 aws-sdk 这样做？我的测试套件无需连接到互联网。
您意识到 aws sdk 没有互联网就什么也做不了，对吧？
@AndrewWei 您需要模拟与元数据服务器的连接。
@AndrewWei 错误信息中已经给你解决方法了:)
在 EC2 服务器上运行时，地址 169.254.169.254 是本地地址，而不是 Internet 地址。如果您想在代码未在 EC2 上运行时将其存根，则错误消息会为您提供执行此操作的确切代码。

【解决方案2】：

这是@helloV 回答的后续。

关于

的完整文档

http://169.254.169.254/latest/meta-data/iam/security-credentials/

实例上的应用程序检索安全凭证由实例元数据项中的角色提供 iam/安全凭证/角色名称。该申请被授予为您定义的操作和资源的权限角色通过与角色关联的安全凭证。这些安全凭证是临时的，我们会自动轮换它们。我们会至少提前五分钟提供新的凭证旧凭据到期。

http://docs.aws.amazon.com/AWSEC2/latest/UserGuide/iam-roles-for-amazon-ec2.html#instance-metadata-security-credentials

【讨论】：

【解决方案3】：

很难研究如何存根。将在这里分享我的发现，希望对其他人有所帮助。

由于 AWS 使用服务名称来获取多种服务的凭证，例如 S3、任何 IAM 角色、SQS 等。但是在处理 IAM 角色时，AWS 会点击 http://169.254.169.254/latest/meta-data/iam/security-credentials/，它会返回附加到机器上的角色名称并使用相同的响应，它会点击 curl http://169.254.169.254/latest/meta-data/iam/security-credentials/#{role}。因此，为了成功模拟 IAM 角色，我们需要对两个 curl 进行存根。下面是相同的代码。

def self.auth(params = {})
  status = params[:status].presence || 200
  url = 'http://169.254.169.254/latest/meta-data/iam/security-credentials/'
  WebMock.stub_request(:get, url)
      .with({headers: request_headers}.compact)
      .to_return(status: status, body: 'SampleRole', headers: {})
end

def self.role_auth(role = 'SampleRole')
  url = "http://169.254.169.254/latest/meta-data/iam/security-credentials/#{role}"
  WebMock.stub_request(:get, url)
      .with({headers: request_headers}.compact)
      .to_return(status: 200, body: sample_response, headers: {})
end

private

def self.request_headers
  {
      'Accept': '*/*',
      'Accept-Encoding': 'gzip;q=1.0,deflate;q=0.6,identity;q=0.3',
      'User-Agent': 'aws-sdk-ruby3/3.21.2'
  }.with_indifferent_access
end

def self.sample_response
  {
      Code: 'Success',
      LastUpdated: '2020-05-08T05:55:23Z',
      Type: 'AWS-HMAC',
      AccessKeyId: Faker::Lorem.characters(10),
      SecretAccessKey: Faker::Lorem.characters(20),
      Token: Faker::Lorem.characters(60),
      Expiration: '2020-05-08T12:00:45Z'
  }.to_json
end

因此，每当任何 rspec 使用 InstanceCredentials 提供程序时，您都需要在每个 rspec 的块之前对上述两个定义的请求进行存根。

例子：

before do
 ApiStub::IamCred.auth
 ApiStub::IamCred.role_auth
end

希望对您有所帮助。快乐编码

【讨论】：

【解决方案4】：

在这个线程中有一些关于如何存根的很好的答案。另一种选择（更好的选择？）是消除 SDK 从实例元数据服务检查凭据的需要。您可以通过提供凭据来做到这一点。这些可能是假的。一个简单的方法是通过 ENV 提供凭证（例如 ENV['AWS_ACCESS_TOKEN'] 和 ENV['SECRET_ACCESS_TOKEN']）。

【讨论】：