Rails ArgumentError: 无效的 %-encoding答案

【问题标题】：Rails ArgumentError: invalid %-encodingRails ArgumentError: 无效的 %-encoding
【发布时间】：2013-04-29 01:11:15
【问题描述】：

上个月，我们有一个机器人定期抓取我们的网站，导致一堆ArgumentError: invalid %-encoding 错误，因为网址格式错误。我查看了机架here 和here 和rails here 中的一堆问题，并查看了this SO thread，但似乎没有明确的解决方案。 GET 错误是否有正确的解决方案？我必须要猴子补丁架吗？

编辑：这是一个回溯：

/usr/local/lib/ruby/1.9.1/uri/common.rb:898:in `decode_www_form_component'
[GEM_ROOT]/gems/rack-1.4.5/lib/rack/utils.rb:41:in `unescape'
[GEM_ROOT]/gems/rack-1.4.5/lib/rack/utils.rb:94:in `block (2 levels) in parse_nested_query'
[GEM_ROOT]/gems/rack-1.4.5/lib/rack/utils.rb:94:in `map'
[GEM_ROOT]/gems/rack-1.4.5/lib/rack/utils.rb:94:in `block in parse_nested_query'
[GEM_ROOT]/gems/rack-1.4.5/lib/rack/utils.rb:93:in `each'
[GEM_ROOT]/gems/rack-1.4.5/lib/rack/utils.rb:93:in `parse_nested_query'
[GEM_ROOT]/gems/rack-1.4.5/lib/rack/request.rb:332:in `parse_query'
[GEM_ROOT]/gems/actionpack-3.2.12/lib/action_dispatch/http/request.rb:269:in `parse_query'
[GEM_ROOT]/gems/rack-1.4.5/lib/rack/request.rb:186:in `GET'
[GEM_ROOT]/gems/actionpack-3.2.12/lib/action_dispatch/http/request.rb:225:in `GET'
[GEM_ROOT]/gems/actionpack-3.2.12/lib/action_dispatch/http/parameters.rb:10:in `parameters'
[GEM_ROOT]/gems/actionpack-3.2.12/lib/action_dispatch/http/filter_parameters.rb:33:in `filtered_parameters'
[GEM_ROOT]/gems/actionpack-3.2.12/lib/action_controller/metal/instrumentation.rb:21:in `process_action'
[GEM_ROOT]/gems/actionpack-3.2.12/lib/action_controller/metal/params_wrapper.rb:207:in `process_action'
[GEM_ROOT]/gems/activerecord-3.2.12/lib/active_record/railties/controller_runtime.rb:18:in `process_action'
[GEM_ROOT]/gems/actionpack-3.2.12/lib/abstract_controller/base.rb:121:in `process'
[GEM_ROOT]/gems/actionpack-3.2.12/lib/abstract_controller/rendering.rb:45:in `process'
[GEM_ROOT]/gems/actionpack-3.2.12/lib/action_controller/metal.rb:203:in `dispatch'
[GEM_ROOT]/gems/actionpack-3.2.12/lib/action_controller/metal/rack_delegation.rb:14:in `dispatch'
[GEM_ROOT]/gems/actionpack-3.2.12/lib/action_controller/metal.rb:246:in `block in action'
[GEM_ROOT]/gems/actionpack-3.2.12/lib/action_dispatch/routing/route_set.rb:73:in `call'
[GEM_ROOT]/gems/actionpack-3.2.12/lib/action_dispatch/routing/route_set.rb:73:in `dispatch'
[GEM_ROOT]/gems/actionpack-3.2.12/lib/action_dispatch/routing/route_set.rb:36:in `call'
[GEM_ROOT]/gems/journey-1.0.4/lib/journey/router.rb:68:in `block in call'
[GEM_ROOT]/gems/journey-1.0.4/lib/journey/router.rb:56:in `each'
[GEM_ROOT]/gems/journey-1.0.4/lib/journey/router.rb:56:in `call'
[GEM_ROOT]/gems/actionpack-3.2.12/lib/action_dispatch/routing/route_set.rb:601:in `call'
[GEM_ROOT]/gems/omniauth-1.1.1/lib/omniauth/strategy.rb:177:in `call!'
[GEM_ROOT]/gems/omniauth-1.1.1/lib/omniauth/strategy.rb:157:in `call'
[GEM_ROOT]/gems/sass-3.2.7/lib/sass/plugin/rack.rb:54:in `call'
[GEM_ROOT]/gems/warden-1.2.1/lib/warden/manager.rb:35:in `block in call'
[GEM_ROOT]/gems/warden-1.2.1/lib/warden/manager.rb:34:in `catch'
[GEM_ROOT]/gems/warden-1.2.1/lib/warden/manager.rb:34:in `call'
[GEM_ROOT]/gems/actionpack-3.2.12/lib/action_dispatch/middleware/best_standards_support.rb:17:in `call'
[GEM_ROOT]/gems/rack-1.4.5/lib/rack/etag.rb:23:in `call'
[GEM_ROOT]/gems/rack-1.4.5/lib/rack/conditionalget.rb:25:in `call'
[GEM_ROOT]/gems/actionpack-3.2.12/lib/action_dispatch/middleware/head.rb:14:in `call'
[GEM_ROOT]/gems/actionpack-3.2.12/lib/action_dispatch/middleware/params_parser.rb:21:in `call'
[GEM_ROOT]/gems/actionpack-3.2.12/lib/action_dispatch/middleware/flash.rb:242:in `call'
[GEM_ROOT]/gems/rack-1.4.5/lib/rack/session/abstract/id.rb:210:in `context'
[GEM_ROOT]/gems/rack-1.4.5/lib/rack/session/abstract/id.rb:205:in `call'
[GEM_ROOT]/gems/actionpack-3.2.12/lib/action_dispatch/middleware/cookies.rb:341:in `call'
[GEM_ROOT]/gems/activerecord-3.2.12/lib/active_record/query_cache.rb:64:in `call'
[GEM_ROOT]/gems/activerecord-3.2.12/lib/active_record/connection_adapters/abstract/connection_pool.rb:479:in `call'
[GEM_ROOT]/gems/actionpack-3.2.12/lib/action_dispatch/middleware/callbacks.rb:28:in `block in call'
[GEM_ROOT]/gems/activesupport-3.2.12/lib/active_support/callbacks.rb:405:in `_run__497203393471184793__call__4495106819278994598__callbacks'
[GEM_ROOT]/gems/activesupport-3.2.12/lib/active_support/callbacks.rb:405:in `__run_callback'
[GEM_ROOT]/gems/activesupport-3.2.12/lib/active_support/callbacks.rb:385:in `_run_call_callbacks'
[GEM_ROOT]/gems/activesupport-3.2.12/lib/active_support/callbacks.rb:81:in `run_callbacks'
[GEM_ROOT]/gems/actionpack-3.2.12/lib/action_dispatch/middleware/callbacks.rb:27:in `call'
[GEM_ROOT]/gems/actionpack-3.2.12/lib/action_dispatch/middleware/remote_ip.rb:31:in `call'
[GEM_ROOT]/gems/actionpack-3.2.12/lib/action_dispatch/middleware/debug_exceptions.rb:16:in `call'
[GEM_ROOT]/gems/actionpack-3.2.12/lib/action_dispatch/middleware/show_exceptions.rb:56:in `call'
[GEM_ROOT]/gems/railties-3.2.12/lib/rails/rack/logger.rb:32:in `call_app'
[GEM_ROOT]/gems/railties-3.2.12/lib/rails/rack/logger.rb:16:in `block in call'
[GEM_ROOT]/gems/activesupport-3.2.12/lib/active_support/tagged_logging.rb:22:in `tagged'
[GEM_ROOT]/gems/railties-3.2.12/lib/rails/rack/logger.rb:16:in `call'
[GEM_ROOT]/gems/actionpack-3.2.12/lib/action_dispatch/middleware/request_id.rb:22:in `call'
[GEM_ROOT]/gems/rack-1.4.5/lib/rack/methodoverride.rb:21:in `call'
[GEM_ROOT]/gems/rack-1.4.5/lib/rack/runtime.rb:17:in `call'
[GEM_ROOT]/gems/activesupport-3.2.12/lib/active_support/cache/strategy/local_cache.rb:72:in `call'
[GEM_ROOT]/gems/rack-1.4.5/lib/rack/lock.rb:15:in `call'
[GEM_ROOT]/gems/rack-cache-1.2/lib/rack/cache/context.rb:136:in `forward'
[GEM_ROOT]/gems/rack-cache-1.2/lib/rack/cache/context.rb:143:in `pass'
[GEM_ROOT]/gems/rack-cache-1.2/lib/rack/cache/context.rb:172:in `rescue in lookup'
[GEM_ROOT]/gems/rack-cache-1.2/lib/rack/cache/context.rb:168:in `lookup'
[GEM_ROOT]/gems/rack-cache-1.2/lib/rack/cache/context.rb:66:in `call!'
[GEM_ROOT]/gems/rack-cache-1.2/lib/rack/cache/context.rb:51:in `call'
[GEM_ROOT]/gems/railties-3.2.12/lib/rails/engine.rb:479:in `call'
[GEM_ROOT]/gems/railties-3.2.12/lib/rails/application.rb:223:in `call'
[GEM_ROOT]/gems/railties-3.2.12/lib/rails/railtie/configurable.rb:30:in `method_missing'
/usr/local/lib/ruby/gems/1.9.1/gems/passenger-3.0.13/lib/phusion_passenger/rack/request_handler.rb:96:in `process_request'
/usr/local/lib/ruby/gems/1.9.1/gems/passenger-3.0.13/lib/phusion_passenger/abstract_request_handler.rb:516:in `accept_and_process_next_request'
/usr/local/lib/ruby/gems/1.9.1/gems/passenger-3.0.13/lib/phusion_passenger/abstract_request_handler.rb:274:in `main_loop'
/usr/local/lib/ruby/gems/1.9.1/gems/passenger-3.0.13/lib/phusion_passenger/rack/application_spawner.rb:206:in `start_request_handler'
/usr/local/lib/ruby/gems/1.9.1/gems/passenger-3.0.13/lib/phusion_passenger/rack/application_spawner.rb:171:in `block in handle_spawn_application'
/usr/local/lib/ruby/gems/1.9.1/gems/passenger-3.0.13/lib/phusion_passenger/utils.rb:479:in `safe_fork'
/usr/local/lib/ruby/gems/1.9.1/gems/passenger-3.0.13/lib/phusion_passenger/rack/application_spawner.rb:166:in `handle_spawn_application'
/usr/local/lib/ruby/gems/1.9.1/gems/passenger-3.0.13/lib/phusion_passenger/abstract_server.rb:357:in `server_main_loop'
/usr/local/lib/ruby/gems/1.9.1/gems/passenger-3.0.13/lib/phusion_passenger/abstract_server.rb:206:in `start_synchronously'
/usr/local/lib/ruby/gems/1.9.1/gems/passenger-3.0.13/lib/phusion_passenger/abstract_server.rb:180:in `start'
/usr/local/lib/ruby/gems/1.9.1/gems/passenger-3.0.13/lib/phusion_passenger/rack/application_spawner.rb:129:in `start'
/usr/local/lib/ruby/gems/1.9.1/gems/passenger-3.0.13/lib/phusion_passenger/spawn_manager.rb:253:in `block (2 levels) in spawn_rack_application'
/usr/local/lib/ruby/gems/1.9.1/gems/passenger-3.0.13/lib/phusion_passenger/abstract_server_collection.rb:132:in `lookup_or_add'
/usr/local/lib/ruby/gems/1.9.1/gems/passenger-3.0.13/lib/phusion_passenger/spawn_manager.rb:246:in `block in spawn_rack_application'
/usr/local/lib/ruby/gems/1.9.1/gems/passenger-3.0.13/lib/phusion_passenger/abstract_server_collection.rb:82:in `block in synchronize'
/usr/local/lib/ruby/gems/1.9.1/gems/passenger-3.0.13/lib/phusion_passenger/abstract_server_collection.rb:79:in `synchronize'
/usr/local/lib/ruby/gems/1.9.1/gems/passenger-3.0.13/lib/phusion_passenger/spawn_manager.rb:244:in `spawn_rack_application'
/usr/local/lib/ruby/gems/1.9.1/gems/passenger-3.0.13/lib/phusion_passenger/spawn_manager.rb:137:in `spawn_application'
/usr/local/lib/ruby/gems/1.9.1/gems/passenger-3.0.13/lib/phusion_passenger/spawn_manager.rb:275:in `handle_spawn_application'
/usr/local/lib/ruby/gems/1.9.1/gems/passenger-3.0.13/lib/phusion_passenger/abstract_server.rb:357:in `server_main_loop'
/usr/local/lib/ruby/gems/1.9.1/gems/passenger-3.0.13/lib/phusion_passenger/abstract_server.rb:206:in `start_synchronously'
/usr/local/lib/ruby/gems/1.9.1/gems/passenger-3.0.13/helper-scripts/passenger-spawn-server:99:in `<main>'

【问题讨论】：

可能在最新的机架版本中得到修复。似乎一些错误，including one I commented on，当我两年前遇到这个问题时，已经被关闭了。阻止机器人是一种选择吗？
遗憾的是，我尝试升级到最新的机架，但仍然发现问题（然后再次降级，因为我不得不关闭其他一些 gem）。它来自多个 IP 地址，因此变成了打地鼠游戏，而且我希望必须有更好的方法。 :)
你能分享回溯吗？
呃。这个问题在我们的应用中也很普遍。

标签： ruby-on-rails ruby-on-rails-3 url

【解决方案1】：

如果您不介意猴子补丁 Rack，请在 config/initializers 文件（例如 rack.rb）中创建以下内容：

module Rack
  module Utils
    if defined?(::Encoding)
      def unescape(s, encoding = Encoding::UTF_8)
        begin
          URI.decode_www_form_component(s, encoding)
        rescue ArgumentError
          URI.decode_www_form_component(URI.encode(s), encoding)
        end
      end
    else
      def unescape(s, encoding = nil)
        begin
          URI.decode_www_form_component(s, encoding)
        rescue ArgumentError
          URI.decode_www_form_component(URI.encode(s), encoding)
        end
      end
    end
    module_function :unescape
  end
end

附言它适用于乘客，但不适用于 Webrick 和 Thin。看起来 webrick 和 Thin 都解析了一个请求，所以失败发生在初始化程序加载之前。例如，thin-1.6.2/lib/thin/request.rb:84 中发生 Thin 错误。

【讨论】：

【解决方案2】：

您可以注入一个旨在检测这些问题并优雅地失败的中间件。基本思想是尝试解析查询字符串，如果失败，则使用 HTTP 400 退出。否则，只允许请求通过。

class RefuseInvalidRequest
  def initialize(app)
    @app = app
  end

  def call(env)
    query = Rack::Utils.parse_nested_query(env['QUERY_STRING'].to_s) rescue :bad_query
    if query == :bad_query
      [400, {'Content-Type' => 'text/plain'}, "Bad Request"]
    else
      @app.call(env)
    end
  end
end

我没有对此进行测试，但这个概念应该可以工作。

【讨论】：

它不适用于我的开发环境。我不知道为什么，但它只在日志中显示!! Invalid request。并尝试在def initialize 上添加puts "called"，它不会在日志中输出called。只有!! Invalid request。是的，我已将 config.middleware.use("RefuseInvalidRequest") 添加到我的 application.rb 中。知道为什么会这样吗？
也许你没有在你的中间件堆栈中注入足够高的中间件。
config.middleware.insert_before Rack::Runtime, "RefuseInvalidRequest"。我可以插入的最高的。实际上是Rack::MiniProfiler 和Honeybadger。但我不能在它之前插入。还是一样的输出。用日志检查我的旧问题stackoverflow.com/questions/21229499/…
我遇到了同样的问题，但是使用 POST 数据（如 stackoverflow.com/questions/15769681/…），我使用了这个修复程序，但是用 request_content = Rack::Request.new(env).POST rescue :bad_form_data 替换了 query = 行，它就像一个魅力.实际上，我还必须将“错误请求”响应主体放入 Ruby 1.9 的数组中，所以：[400, headers, ["Bad Request"]]