乘客并发连接错误

Question

我有生产服务器（Nginx + Passenger）。当我尝试从另一台计算机 ab -n 3 -c 3 myhost.ru/ 时，我在我的 nginx error.log 中收到此错误日志：

[ pid=21160 thr=139775297914624 file=ext/nginx/HelperAgent.cpp:584 time=2011-08-31 15:25:49.22 ]: Uncaught exception in PassengerServer client thread:
   exception: Cannot read response from backend process: Connection reset by peer (104)
   backtrace:
     in 'void Client::forwardResponse(Passenger::SessionPtr&, Passenger::FileDescriptor&, const Passenger::AnalyticsLogPtr&)' (HelperAgent.cpp:330)
     in 'void Client::handleRequest(Passenger::FileDescriptor&)' (HelperAgent.cpp:510)
     in 'void Client::threadMain()' (HelperAgent.cpp:603)

[ pid=21249 thr=15884760 file=utils.rb:176 time=2011-08-31 15:25:49.022 ]: *** Exception PGError in application (server closed the connection unexpectedly
  This probably means the server terminated abnormally
  before or while processing the request.
) (process 21249, thread #<Thread:0x00000001e4c3b0>):
  from /home/deployer/yandex.ru/shared/bundle/ruby/1.9.1/gems/activerecord-3.1.0/lib/active_record/connection_adapters/postgresql_adapter.rb:276:in `exec'
  from /home/deployer/yandex.ru/shared/bundle/ruby/1.9.1/gems/activerecord-3.1.0/lib/active_record/connection_adapters/postgresql_adapter.rb:276:in `block in clear_cache!'
  from /home/deployer/yandex.ru/shared/bundle/ruby/1.9.1/gems/activerecord-3.1.0/lib/active_record/connection_adapters/postgresql_adapter.rb:275:in `each_value'
  from /home/deployer/yandex.ru/shared/bundle/ruby/1.9.1/gems/activerecord-3.1.0/lib/active_record/connection_adapters/postgresql_adapter.rb:275:in `clear_cache!'
  from /home/deployer/yandex.ru/shared/bundle/ruby/1.9.1/gems/activerecord-3.1.0/lib/active_record/connection_adapters/postgresql_adapter.rb:303:in `disconnect!'
  from /home/deployer/yandex.ru/shared/bundle/ruby/1.9.1/gems/activerecord-3.1.0/lib/active_record/connection_adapters/abstract/connection_pool.rb:202:in `block in disconnect!'
  from /home/deployer/yandex.ru/shared/bundle/ruby/1.9.1/gems/activerecord-3.1.0/lib/active_record/connection_adapters/abstract/connection_pool.rb:201:in `each'
  from /home/deployer/yandex.ru/shared/bundle/ruby/1.9.1/gems/activerecord-3.1.0/lib/active_record/connection_adapters/abstract/connection_pool.rb:201:in `disconnect!'
  from /home/deployer/yandex.ru/shared/bundle/ruby/1.9.1/gems/activesupport-3.1.0/lib/active_support/core_ext/module/synchronization.rb:35:in `block in disconnect_with_synchronization!'
  from /home/deployer/.rvm/rubies/ruby-1.9.2-p180/lib/ruby/1.9.1/monitor.rb:201:in `mon_synchronize'
  from /home/deployer/yandex.ru/shared/bundle/ruby/1.9.1/gems/activesupport-3.1.0/lib/active_support/core_ext/module/synchronization.rb:34:in `disconnect_with_synchronization!'
  from /home/deployer/yandex.ru/shared/bundle/ruby/1.9.1/gems/activerecord-3.1.0/lib/active_record/connection_adapters/abstract/connection_pool.rb:395:in `block in clear_all_connections!'
  from /home/deployer/yandex.ru/shared/bundle/ruby/1.9.1/gems/activerecord-3.1.0/lib/active_record/connection_adapters/abstract/connection_pool.rb:395:in `each_value'
  from /home/deployer/yandex.ru/shared/bundle/ruby/1.9.1/gems/activerecord-3.1.0/lib/active_record/connection_adapters/abstract/connection_pool.rb:395:in `clear_all_connections!'
  from /home/deployer/yandex.ru/shared/bundle/ruby/1.9.1/gems/activerecord-3.1.0/lib/active_record/connection_adapters/abstract/connection_specification.rb:123:in `clear_all_connections!'
  from /home/deployer/.rvm/gems/ruby-1.9.2-p180/gems/passenger-3.0.7/lib/phusion_passenger/utils.rb:398:in `before_handling_requests'
  from /home/deployer/.rvm/gems/ruby-1.9.2-p180/gems/passenger-3.0.7/lib/phusion_passenger/rack/application_spawner.rb:204:in `start_request_handler'
  from /home/deployer/.rvm/gems/ruby-1.9.2-p180/gems/passenger-3.0.7/lib/phusion_passenger/rack/application_spawner.rb:170:in `block in handle_spawn_application'
  from /home/deployer/.rvm/gems/ruby-1.9.2-p180/gems/passenger-3.0.7/lib/phusion_passenger/utils.rb:479:in `safe_fork'
  from /home/deployer/.rvm/gems/ruby-1.9.2-p180/gems/passenger-3.0.7/lib/phusion_passenger/rack/application_spawner.rb:165:in `handle_spawn_application'
  from /home/deployer/.rvm/gems/ruby-1.9.2-p180/gems/passenger-3.0.7/lib/phusion_passenger/abstract_server.rb:357:in `server_main_loop'
  from /home/deployer/.rvm/gems/ruby-1.9.2-p180/gems/passenger-3.0.7/lib/phusion_passenger/abstract_server.rb:206:in `start_synchronously'
  from /home/deployer/.rvm/gems/ruby-1.9.2-p180/gems/passenger-3.0.7/lib/phusion_passenger/abstract_server.rb:180:in `start'
  from /home/deployer/.rvm/gems/ruby-1.9.2-p180/gems/passenger-3.0.7/lib/phusion_passenger/rack/application_spawner.rb:128:in `start'
  from /home/deployer/.rvm/gems/ruby-1.9.2-p180/gems/passenger-3.0.7/lib/phusion_passenger/spawn_manager.rb:253:in `block (2 levels) in spawn_rack_application'
  from /home/deployer/.rvm/gems/ruby-1.9.2-p180/gems/passenger-3.0.7/lib/phusion_passenger/abstract_server_collection.rb:132:in `lookup_or_add'
  from /home/deployer/.rvm/gems/ruby-1.9.2-p180/gems/passenger-3.0.7/lib/phusion_passenger/spawn_manager.rb:246:in `block in spawn_rack_application'
  from /home/deployer/.rvm/gems/ruby-1.9.2-p180/gems/passenger-3.0.7/lib/phusion_passenger/abstract_server_collection.rb:82:in `block in synchronize'
  from <internal:prelude>:10:in `synchronize'
  from /home/deployer/.rvm/gems/ruby-1.9.2-p180/gems/passenger-3.0.7/lib/phusion_passenger/abstract_server_collection.rb:79:in `synchronize'
  from /home/deployer/.rvm/gems/ruby-1.9.2-p180/gems/passenger-3.0.7/lib/phusion_passenger/spawn_manager.rb:244:in `spawn_rack_application'
  from /home/deployer/.rvm/gems/ruby-1.9.2-p180/gems/passenger-3.0.7/lib/phusion_passenger/spawn_manager.rb:137:in `spawn_application'
  from /home/deployer/.rvm/gems/ruby-1.9.2-p180/gems/passenger-3.0.7/lib/phusion_passenger/spawn_manager.rb:275:in `handle_spawn_application'
  from /home/deployer/.rvm/gems/ruby-1.9.2-p180/gems/passenger-3.0.7/lib/phusion_passenger/abstract_server.rb:357:in `server_main_loop'
  from /home/deployer/.rvm/gems/ruby-1.9.2-p180/gems/passenger-3.0.7/lib/phusion_passenger/abstract_server.rb:206:in `start_synchronously'
  from /home/deployer/.rvm/gems/ruby-1.9.2-p180/gems/passenger-3.0.7/helper-scripts/passenger-spawn-server:99:in `<main>'
2011/08/31 15:25:49 [error] 21197#0: *41173 upstream prematurely closed connection while reading response header from upstream, client: 78.47.253.164, server: yandex.ru, request: "GET / HTTP/1.0", upstream: "passenger:unix:/passenger_helper_server:", host: "yandex.ru"
[ pid=21160 thr=139775298115328 file=ext/nginx/HelperAgent.cpp:584 time=2011-08-31 15:25:49.31 ]: Uncaught exception in PassengerServer client thread:
   exception: Cannot read response from backend process: Connection reset by peer (104)
   backtrace:
     in 'void Client::forwardResponse(Passenger::SessionPtr&, Passenger::FileDescriptor&, const Passenger::AnalyticsLogPtr&)' (HelperAgent.cpp:330)
     in 'void Client::handleRequest(Passenger::FileDescriptor&)' (HelperAgent.cpp:510)
     in 'void Client::threadMain()' (HelperAgent.cpp:603)

2011/08/31 15:25:49 [error] 21197#0: *41172 upstream prematurely closed connection while reading response header from upstream, client: 78.47.253.164, server: yandex.ru, request: "GET / HTTP/1.0", upstream: "passenger:unix:/passenger_helper_server:", host: "yandex.ru"
[ pid=21257 thr=15884760 file=utils.rb:176 time=2011-08-31 15:25:49.032 ]: *** Exception PGError in application (server closed the connection unexpectedly
  This probably means the server terminated abnormally
  before or while processing the request.
) (process 21257, thread #<Thread:0x00000001e4c3b0>):
  from /home/deployer/yandex.ru/shared/bundle/ruby/1.9.1/gems/activerecord-3.1.0/lib/active_record/connection_adapters/postgresql_adapter.rb:276:in `exec'
  from /home/deployer/yandex.ru/shared/bundle/ruby/1.9.1/gems/activerecord-3.1.0/lib/active_record/connection_adapters/postgresql_adapter.rb:276:in `block in clear_cache!'
  from /home/deployer/yandex.ru/shared/bundle/ruby/1.9.1/gems/activerecord-3.1.0/lib/active_record/connection_adapters/postgresql_adapter.rb:275:in `each_value'
  from /home/deployer/yandex.ru/shared/bundle/ruby/1.9.1/gems/activerecord-3.1.0/lib/active_record/connection_adapters/postgresql_adapter.rb:275:in `clear_cache!'
  from /home/deployer/yandex.ru/shared/bundle/ruby/1.9.1/gems/activerecord-3.1.0/lib/active_record/connection_adapters/postgresql_adapter.rb:303:in `disconnect!'
  from /home/deployer/yandex.ru/shared/bundle/ruby/1.9.1/gems/activerecord-3.1.0/lib/active_record/connection_adapters/abstract/connection_pool.rb:202:in `block in disconnect!'
  from /home/deployer/yandex.ru/shared/bundle/ruby/1.9.1/gems/activerecord-3.1.0/lib/active_record/connection_adapters/abstract/connection_pool.rb:201:in `each'
  from /home/deployer/yandex.ru/shared/bundle/ruby/1.9.1/gems/activerecord-3.1.0/lib/active_record/connection_adapters/abstract/connection_pool.rb:201:in `disconnect!'
  from /home/deployer/yandex.ru/shared/bundle/ruby/1.9.1/gems/activesupport-3.1.0/lib/active_support/core_ext/module/synchronization.rb:35:in `block in disconnect_with_synchronization!'
  from /home/deployer/.rvm/rubies/ruby-1.9.2-p180/lib/ruby/1.9.1/monitor.rb:201:in `mon_synchronize'
  from /home/deployer/yandex.ru/shared/bundle/ruby/1.9.1/gems/activesupport-3.1.0/lib/active_support/core_ext/module/synchronization.rb:34:in `disconnect_with_synchronization!'
  from /home/deployer/yandex.ru/shared/bundle/ruby/1.9.1/gems/activerecord-3.1.0/lib/active_record/connection_adapters/abstract/connection_pool.rb:395:in `block in clear_all_connections!'
  from /home/deployer/yandex.ru/shared/bundle/ruby/1.9.1/gems/activerecord-3.1.0/lib/active_record/connection_adapters/abstract/connection_pool.rb:395:in `each_value'
  from /home/deployer/yandex.ru/shared/bundle/ruby/1.9.1/gems/activerecord-3.1.0/lib/active_record/connection_adapters/abstract/connection_pool.rb:395:in `clear_all_connections!'
  from /home/deployer/yandex.ru/shared/bundle/ruby/1.9.1/gems/activerecord-3.1.0/lib/active_record/connection_adapters/abstract/connection_specification.rb:123:in `clear_all_connections!'
  from /home/deployer/.rvm/gems/ruby-1.9.2-p180/gems/passenger-3.0.7/lib/phusion_passenger/utils.rb:398:in `before_handling_requests'
  from /home/deployer/.rvm/gems/ruby-1.9.2-p180/gems/passenger-3.0.7/lib/phusion_passenger/rack/application_spawner.rb:204:in `start_request_handler'
  from /home/deployer/.rvm/gems/ruby-1.9.2-p180/gems/passenger-3.0.7/lib/phusion_passenger/rack/application_spawner.rb:170:in `block in handle_spawn_application'
  from /home/deployer/.rvm/gems/ruby-1.9.2-p180/gems/passenger-3.0.7/lib/phusion_passenger/utils.rb:479:in `safe_fork'
  from /home/deployer/.rvm/gems/ruby-1.9.2-p180/gems/passenger-3.0.7/lib/phusion_passenger/rack/application_spawner.rb:165:in `handle_spawn_application'
  from /home/deployer/.rvm/gems/ruby-1.9.2-p180/gems/passenger-3.0.7/lib/phusion_passenger/abstract_server.rb:357:in `server_main_loop'
  from /home/deployer/.rvm/gems/ruby-1.9.2-p180/gems/passenger-3.0.7/lib/phusion_passenger/abstract_server.rb:206:in `start_synchronously'
  from /home/deployer/.rvm/gems/ruby-1.9.2-p180/gems/passenger-3.0.7/lib/phusion_passenger/abstract_server.rb:180:in `start'
  from /home/deployer/.rvm/gems/ruby-1.9.2-p180/gems/passenger-3.0.7/lib/phusion_passenger/rack/application_spawner.rb:128:in `start'
  from /home/deployer/.rvm/gems/ruby-1.9.2-p180/gems/passenger-3.0.7/lib/phusion_passenger/spawn_manager.rb:253:in `block (2 levels) in spawn_rack_application'
  from /home/deployer/.rvm/gems/ruby-1.9.2-p180/gems/passenger-3.0.7/lib/phusion_passenger/abstract_server_collection.rb:132:in `lookup_or_add'
  from /home/deployer/.rvm/gems/ruby-1.9.2-p180/gems/passenger-3.0.7/lib/phusion_passenger/spawn_manager.rb:246:in `block in spawn_rack_application'
  from /home/deployer/.rvm/gems/ruby-1.9.2-p180/gems/passenger-3.0.7/lib/phusion_passenger/abstract_server_collection.rb:82:in `block in synchronize'
  from <internal:prelude>:10:in `synchronize'
  from /home/deployer/.rvm/gems/ruby-1.9.2-p180/gems/passenger-3.0.7/lib/phusion_passenger/abstract_server_collection.rb:79:in `synchronize'
  from /home/deployer/.rvm/gems/ruby-1.9.2-p180/gems/passenger-3.0.7/lib/phusion_passenger/spawn_manager.rb:244:in `spawn_rack_application'
  from /home/deployer/.rvm/gems/ruby-1.9.2-p180/gems/passenger-3.0.7/lib/phusion_passenger/spawn_manager.rb:137:in `spawn_application'
  from /home/deployer/.rvm/gems/ruby-1.9.2-p180/gems/passenger-3.0.7/lib/phusion_passenger/spawn_manager.rb:275:in `handle_spawn_application'
  from /home/deployer/.rvm/gems/ruby-1.9.2-p180/gems/passenger-3.0.7/lib/phusion_passenger/abstract_server.rb:357:in `server_main_loop'
  from /home/deployer/.rvm/gems/ruby-1.9.2-p180/gems/passenger-3.0.7/lib/phusion_passenger/abstract_server.rb:206:in `start_synchronously'
  from /home/deployer/.rvm/gems/ruby-1.9.2-p180/gems/passenger-3.0.7/helper-scripts/passenger-spawn-server:99:in `<main>'

Answer 1

您是否正在分配或管理自己的数据库连接/池？

Passenger 在分叉后为每个后台工作人员重新连接默认的 ActiveRecord 数据库池。

但是，其他资源或手动管理的数据库连接需要在 fork 后为每个 worker 手动设置/重新连接。

我写了一篇详细的博客文章来解释如何解决这个问题：http://blog.winfieldpeterson.com/2011/01/06/passenger-resource-collision/

这听起来就像您正在经历的那样，多个工作人员试图共享一个数据库连接并由于资源争用而在负载下产生错误。

Answer 2

我自己找到了解决方案。还有passenger_spawn_method conservative必须写成nginx.conf而不是默认的passenger_spawn_method conservative smart-lv2。

Answer 3

我们在使用 Rails 3.1、pg、ruby 1.9.3 和使用 Apache + Passenger 3.0.9 时遇到了同样的问题。升级到 Rails 3.1.1 似乎可以解决这个问题。

https://github.com/rails/rails/issues/1339