从 Github 下载 Chef remote_file 失败

Question

我遇到的问题是，使用 remote_file 资源从 Github 下载失败在一个特定的 Chef 节点上（无论是在正常 chef-client 模式下，还是在 chef-shell 下运行时） .

使用wget 和curl 从同一台服务器下载。同样的资源声明也适用于我的机器。

资源声明（最简单的失败版本）：

远程文件“/var/gerrit/review/plugins/rabbitmq.jar”做 来源“https://github.com/TYPO3-infrastructure/gerrit-rabbitmq-plugin/releases/download/rabbitmq-1.5-SNAPSHOT-20150203154700/rabbitmq-1.5-SNAPSHOT-20150203154700.jar” 结尾

我测试过的来自 Github 的 S3 存储桶的所有其他发布文件也是如此。虽然403 Forbidden当然会提醒我速率限制等。我不明白为什么当curl从同一个IP下载时会出现这种情况。

完整的调试日志：

* remote_file[/var/gerrit/review/plugins/rabbitmq.jar] 操作创建[2015-10-19T22:40:38+02:00] 信息：处理 remote_file[/var/gerrit/review/plugins/rabbitmq.jar ] action create (site-reviewtypo3org::amqp-publisher line 1) [2015-10-19T22:40:38+02:00] 调试：remote_file[/var/gerrit/review/plugins/rabbitmq.jar] 校验和文件位于 /var/gerrit/review/plugins/rabbitmq.jar。 [2015-10-19T22:40:38+02:00] 调试：remote_file[/var/gerrit/review/plugins/rabbitmq.jar] 检查更改 [2015-10-19T22:40:38+02:00] 调试：缓存控制标头：{"if-modified-since"=>"Tue, 03 Feb 2015 14:57:25 GMT", "if-none-匹配"=>"\"6e60d332ee7e32e1982ed36181769c8f\""} [2015-10-19T22:40:38+02:00] 调试：Chef::HTTP 调用 Chef::HTTP::Decompressor#handle_request [2015-10-19T22:40:38+02:00] 调试：Chef::HTTP 调用 Chef::HTTP::CookieManager#handle_request [2015-10-19T22:40:38+02:00] 调试：Chef::HTTP 调用 Chef::HTTP::ValidateContentLength#handle_request [2015-10-19T22:40:38+02:00] 调试：启动 GET 到 https://github.com/TYPO3-infrastructure/gerrit-rabbitmq-plugin/releases/download/rabbitmq-1.5-SNAPSHOT-20150203154700/ rabbitmq-1.5-SNAPSHOT-20150203154700.jar [2015-10-19T22:40:38+02:00] 调试：---- HTTP 请求标头数据：---- [2015-10-19T22:40:38+02:00] 调试：if-modified-since: 2015 年 2 月 3 日星期二 14:57:25 GMT [2015-10-19T22:40:38+02:00] 调试：如果没有匹配：“6e60d332ee7e32e1982ed36181769c8f” [2015-10-19T22:40:38+02:00] 调试：接受编码：gzip;q=1.0,deflate;q=0.6,identity;q=0.3 [2015-10-19T22:40:38+02:00] 调试：----结束 HTTP 请求标头数据---- [2015-10-19T22:40:39+02:00] 调试：---- HTTP 状态和标头数据：---- [2015-10-19T22:40:39+02:00] 调试：找到 HTTP 1.1 302 [2015-10-19T22:40:39+02:00] 调试：服务器：GitHub.com [2015-10-19T22:40:39+02:00] 调试：日期：2015 年 10 月 19 日星期一 20:40:39 GMT [2015-10-19T22:40:39+02:00] 调试：内容类型：文本/html；字符集=utf-8 [2015-10-19T22:40:39+02:00] 调试：传输编码：分块 [2015-10-19T22:40:39+02:00] 调试：连接：关闭 [2015-10-19T22:40:39+02:00] 调试：状态：找到 302 [2015-10-19T22:40:39+02:00] 调试：内容安全策略：默认源代码 *；脚本-src assets-cdn.github.com;对象-src assets-cdn.github.com; style-src 'self' 'unsafe-inline' 'unsafe-eval' assets-cdn.github.com; img-src 'self' 数据： assets-cdn.github.com identicons.github.com www.google-analytics.com checkout.paypal.com collector.githubapp.com *.githubusercontent.com *.gravatar.com *.wp .com;媒体源“无”； frame-src 'self' render.githubusercontent.com gist.github.com www.youtube.com player.vimeo.com checkout.paypal.com; font-src assets-cdn.github.com; connect-src 'self' live.github.com wss://live.github.com uploads.github.com status.github.com api.github.com www.google-analytics.com api.braintreegateway.com client-analytics .braintreegateway.com github-cloud.s3.amazonaws.com； base-uri '自我'; form-action 'self' github.com gist.github.com [2015-10-19T22:40:39+02:00] 调试：公钥引脚：max-age=300； pin-sha256="WoiWRyIOVNa9ihaBciRSC7XHjliYS9VwUGOIud4PB18="; pin-sha256="JbQbUG5JMJUoI6brnx0x3vZF6jilxsapbXGVfjhN8Fg=";包括子域 [2015-10-19T22:40:39+02:00] 调试：缓存控制：无缓存 [2015-10-19T22:40:39+02:00] 调试：变化：X-PJAX，接受编码 [2015-10-19T22:40:39+02:00] 调试：位置：https://github-cloud.s3.amazonaws.com/releases/30247485/2a842826-abbc-11e4-8a77-45fa1f4254e4.jar?X -Amz-算法= AWS4-HMAC-SHA256＆X-AMZ-凭证= AKIAISTNZFOVBIJMK3TQ％2F20151019％2Fus - 东 - 1％2Fs3％2Faws4_request＆X-AMZ-日期= 20151019T204039Z＆X-AMZ-过期= 300＆X-AMZ-签名= 5263c7cf5f4e0b4dad75ab6b61cab79c4ba52006bc74dddf123455a6806dbc8f＆X-AMZ-SignedHeaders =host&actor_id=0&response-content-disposition=attachment%3B%20filename%3Drabbitmq-1.5-SNAPSHOT-20150203154700.jar&response-content-type=application%2Foctet-stream [2015-10-19T22:40:39+02:00] 调试：x-ua 兼容：IE=Edge，chrome=1 [2015-10-19T22:40:39+02:00] 调试：设置 cookie：logged_in=no；域=.github.com；路径=/;过期=2035 年 10 月 19 日星期五 20:40:39 -0000；安全的;仅Http，_gh_sess = eyJzZXNzaW9uX2lkIjoiYjlhYTBlYzdiZjVjMjRiYjcwOTliMGNmMGQxYTRhODEiLCJzcHlfcmVwbyI6IlRZUE8zLWluZnJhc3RydWN0dXJlL2dlcnJpdC1yYWJiaXRtcS1wbHVnaW4iLCJzcHlfcmVwb19hdCI6MTQ0NTI4NzIzOX0％3D - adb8b6278bfd7dafb550b7ad81d7d00134ec8d28;路径=/;安全的; HttpOnly [2015-10-19T22:40:39+02:00] 调试：x-request-id：3af146b73f99624eb12afb99db1e5ec6 [2015-10-19T22:40:39+02:00] 调试：x 运行时：0.015264 [2015-10-19T22:40:39+02:00] 调试：x-github-request-id:5BB8230D:7D29:1F164E09:56255547 [2015-10-19T22:40:39+02:00] 调试：严格传输安全性：max-age=31536000；包括子域；预载 [2015-10-19T22:40:39+02:00] 调试：x-content-type-options: nosniff [2015-10-19T22:40:39+02:00] 调试：x-xss-保护：1；模式=块 [2015-10-19T22:40:39+02:00] 调试：x-frame-options：拒绝 [2015-10-19T22:40:39+02:00] 调试：x 服务者：a4e77e68361b6b21519d8422c6815f62 [2015-10-19T22:40:39+02:00] 调试：----结束 HTTP 状态/标头数据---- [2015-10-19T22:40:39+02:00] 调试：Chef::HTTP 调用 Chef::HTTP::ValidateContentLength#handle_stream_complete [2015-10-19T22:40:39+02:00] 调试：没有为流式下载收集内容长度信息，无法识别流式下载。 [2015-10-19T22:40:39+02:00] 调试：Chef::HTTP 调用 Chef::HTTP::CookieManager#handle_stream_complete [2015-10-19T22:40:39+02:00] 调试：Chef::HTTP 调用 Chef::HTTP::Decompressor#handle_stream_complete [2015-10-19T22:40:39+02:00] 调试：以下重定向 1/10 [2015-10-19T22:40:39+02:00] 调试：启动 GET 到 https://github-cloud.s3.amazonaws.com/releases/30247485/2a842826-abbc-11e4-8a77-45fa1f4254e4.jar？ X-AMZ-算法= AWS4-HMAC-SHA256＆X-AMZ-凭证= AKIAISTNZFOVBIJMK3TQ％2F20151019％2Fus - 东 - 1％2Fs3％2Faws4_request＆X-AMZ-日期= 20151019T204039Z＆X-AMZ-过期= 300＆X-AMZ-签名= 5263c7cf5f4e0b4dad75ab6b61cab79c4ba52006bc74dddf123455a6806dbc8f＆X-Amz- SignedHeaders=host&actor_id=0&response-content-disposition=attachment%3B%20filename%3Drabbitmq-1.5-SNAPSHOT-20150203154700.jar&response-content-type=application%2Foctet-stream [2015-10-19T22:40:39+02:00] 调试：---- HTTP 请求标头数据：---- [2015-10-19T22:40:39+02:00] 调试：if-modified-since：2015 年 2 月 3 日星期二 14:57:25 GMT [2015-10-19T22:40:39+02:00] 调试：如果没有匹配：“6e60d332ee7e32e1982ed36181769c8f” [2015-10-19T22:40:39+02:00] 调试：接受编码：gzip;q=1.0,deflate;q=0.6,identity;q=0.3 [2015-10-19T22:40:39+02:00] 调试：----结束 HTTP 请求标头数据---- [2015-10-19T22:40:40+02:00] 调试：---- HTTP 状态和标头数据：---- [2015-10-19T22:40:40+02:00] 调试：HTTP 1.1 403 禁止 [2015-10-19T22:40:40+02:00] 调试：x-amz-request-id：1EF201092B540030 [2015-10-19T22:40:40+02:00] 调试：x-amz-id-2：JqQbWTxoW4fokAYVIdboCXVPQTxIFndFF+z3du8iim/r+cDBYaWcXyC08gHOeiV1 [2015-10-19T22:40:40+02:00] 调试：内容类型：应用程序/xml [2015-10-19T22:40:40+02:00] 调试：传输编码：分块 [2015-10-19T22:40:40+02:00] 调试：日期：2015 年 10 月 19 日星期一 20:40:39 GMT [2015-10-19T22:40:40+02:00] 调试：服务器：AmazonS3 [2015-10-19T22:40:40+02:00] 调试：连接：关闭 [2015-10-19T22:40:40+02:00] 调试：----结束 HTTP 状态/标头数据---- [2015-10-19T22:40:40+02:00] 调试：Chef::HTTP 调用 Chef::HTTP::ValidateContentLength#handle_stream_complete [2015-10-19T22:40:40+02:00] 调试：没有为流式下载收集内容长度信息，无法识别流式下载。 [2015-10-19T22:40:40+02:00] 调试：Chef::HTTP 调用 Chef::HTTP::CookieManager#handle_stream_complete [2015-10-19T22:40:40+02:00] 调试：Chef::HTTP 调用 Chef::HTTP::Decompressor#handle_stream_complete [2015-10-19T22:40:40+02:00] 信息：HTTP 请求返回 403 禁止： ==================================================== =============================== 对资源“remote_file[/var/gerrit/review/plugins/rabbitmq.jar]”执行操作“create”时出错 ==================================================== =============================== 网络::HTTPServerException ---------------------- 403“禁止” 资源声明： --------------------- # 在 /var/chef/cache/cookbooks/site-reviewtypo3org/recipes/amqp-publisher.rb 1：remote_file "#{node['gerrit']['install_dir']}/plugins/rabbitmq.jar" 做 2：来源“https://github.com/TYPO3-infrastructure/gerrit-rabbitmq-plugin/releases/download/rabbitmq-1.5-SNAPSHOT-20150203154700/rabbitmq-1.5-SNAPSHOT-20150203154700.jar” 3：所有者节点['gerrit']['user'] 4：组节点['gerrit']['group'] 5：结束 6： 编译资源： ------------------ # 在 /var/chef/cache/cookbooks/site-reviewtypo3org/recipes/amqp-publisher.rb:1:in `from_file' 中声明 remote_file("/var/gerrit/review/plugins/rabbitmq.jar") 做 提供者 Chef::Provider::RemoteFile 动作“创造” 重试 0 重试延迟 2 守卫解释器：默认 路径“/var/gerrit/review/plugins/rabbitmq.jar” 备份 5 atomic_update 真 来源 [“https://github.com/TYPO3-infrastructure/gerrit-rabbitmq-plugin/releases/download/rabbitmq-1.5-SNAPSHOT-20150203154700/rabbitmq-1.5-SNAPSHOT-20150203154700.jar”] use_etag 真 use_last_modified 真 食谱名称“网站-reviewtypo3org” recipe_name "amqp-发布者" 所有者“格里特” “格里特”组 结尾 [2015-10-19T22:40:40+02:00] 信息：在重新引发异常之前运行排队的延迟通知 [2015-10-19T22:40:40+02:00] 调试：重新引发异常：Net::HTTPServerException - remote_file[/var/gerrit/review/plugins/rabbitmq.jar] (site-reviewtypo3org::amqp-发布者行 1) 出现错误：Net::HTTPServerException: 403 "Forbidden" /opt/chef/embedded/lib/ruby/1.9.1/net/http.rb:2633:in `错误！ /opt/chef/embedded/lib/ruby/gems/1.9.1/gems/chef-11.12.2/lib/chef/http.rb:177:in `streaming_request' /opt/chef/embedded/lib/ruby/gems/1.9.1/gems/chef-11.12.2/lib/chef/provider/remote_file/http.rb:60:in `fetch' /opt/chef/embedded/lib/ruby/gems/1.9.1/gems/chef-11.12.2/lib/chef/provider/remote_file/content.rb:65:in `grab_file_from_uri' /opt/chef/embedded/lib/ruby/gems/1.9.1/gems/chef-11.12.2/lib/chef/provider/remote_file/content.rb:50:in `try_multiple_sources' /opt/chef/embedded/lib/ruby/gems/1.9.1/gems/chef-11.12.2/lib/chef/provider/remote_file/content.rb:39:in `file_for_provider' /opt/chef/embedded/lib/ruby/gems/1.9.1/gems/chef-11.12.2/lib/chef/file_content_management/content_base.rb:40:in `tempfile' /opt/chef/embedded/lib/ruby/gems/1.9.1/gems/chef-11.12.2/lib/chef/provider/file.rb:411:in `tempfile' /opt/chef/embedded/lib/ruby/gems/1.9.1/gems/chef-11.12.2/lib/chef/provider/file.rb:350:in `do_contents_changes' /opt/chef/embedded/lib/ruby/gems/1.9.1/gems/chef-11.12.2/lib/chef/provider/file.rb:121:in `action_create' /opt/chef/embedded/lib/ruby/gems/1.9.1/gems/chef-11.12.2/lib/chef/provider.rb:120:in `run_action' /opt/chef/embedded/lib/ruby/gems/1.9.1/gems/chef-11.12.2/lib/chef/resource.rb:637:in `run_action' /opt/chef/embedded/lib/ruby/gems/1.9.1/gems/chef-11.12.2/lib/chef/runner.rb:49:in `run_action' /opt/chef/embedded/lib/ruby/gems/1.9.1/gems/chef-11.12.2/lib/chef/runner.rb:81:in `block (2 levels) in converge' /opt/chef/embedded/lib/ruby/gems/1.9.1/gems/chef-11.12.2/lib/chef/runner.rb:81:in `each' /opt/chef/embedded/lib/ruby/gems/1.9.1/gems/chef-11.12.2/lib/chef/runner.rb:81:in `块聚合' /opt/chef/embedded/lib/ruby/gems/1.9.1/gems/chef-11.12.2/lib/chef/resource_collection.rb:98:in `block in execute_each_resource' /opt/chef/embedded/lib/ruby/gems/1.9.1/gems/chef-11.12.2/lib/chef/resource_collection/stepable_iterator.rb:116:in `call' /opt/chef/embedded/lib/ruby/gems/1.9.1/gems/chef-11.12.2/lib/chef/resource_collection/stepable_iterator.rb:116:in `call_iterator_block' /opt/chef/embedded/lib/ruby/gems/1.9.1/gems/chef-11.12.2/lib/chef/resource_collection/stepable_iterator.rb:85:in `step' /opt/chef/embedded/lib/ruby/gems/1.9.1/gems/chef-11.12.2/lib/chef/resource_collection/stepable_iterator.rb:104：在“迭代”中 /opt/chef/embedded/lib/ruby/gems/1.9.1/gems/chef-11.12.2/lib/chef/resource_collection/stepable_iterator.rb:55:in `each_with_index' /opt/chef/embedded/lib/ruby/gems/1.9.1/gems/chef-11.12.2/lib/chef/resource_collection.rb:96:in `execute_each_resource' /opt/chef/embedded/lib/ruby/gems/1.9.1/gems/chef-11.12.2/lib/chef/runner.rb:80:in `converge' /opt/chef/embedded/lib/ruby/gems/1.9.1/gems/chef-11.12.2/lib/chef/client.rb:345:in `converge' /opt/chef/embedded/lib/ruby/gems/1.9.1/gems/chef-11.12.2/lib/chef/client.rb:431:in `do_run' /opt/chef/embedded/lib/ruby/gems/1.9.1/gems/chef-11.12.2/lib/chef/client.rb:213:in `block in run' /opt/chef/embedded/lib/ruby/gems/1.9.1/gems/chef-11.12.2/lib/chef/client.rb:207:in `fork' /opt/chef/embedded/lib/ruby/gems/1.9.1/gems/chef-11.12.2/lib/chef/client.rb:207:in `run' /opt/chef/embedded/lib/ruby/gems/1.9.1/gems/chef-11.12.2/lib/chef/application.rb:217:in `run_chef_client' /opt/chef/embedded/lib/ruby/gems/1.9.1/gems/chef-11.12.2/lib/chef/application/client.rb:328:in `block in run_application' /opt/chef/embedded/lib/ruby/gems/1.9.1/gems/chef-11.12.2/lib/chef/application/client.rb:317:in `loop' /opt/chef/embedded/lib/ruby/gems/1.9.1/gems/chef-11.12.2/lib/chef/application/client.rb:317:in `run_application' /opt/chef/embedded/lib/ruby/gems/1.9.1/gems/chef-11.12.2/lib/chef/application.rb:67:in `run' /opt/chef/embedded/lib/ruby/gems/1.9.1/gems/chef-11.12.2/bin/chef-client:26:in `' /usr/bin/chef-client:23:in `load' /usr/bin/chef-client:23:in `'

Answer 1

我能够通过升级到新版本的 Chef 来解决我的问题。

版本 11.14.2 修复了这个问题（虽然我无法弄清楚 ChangeLog，它实际上是变化的）。