Heroku dev 上的 Postgres 进程分叉错误

Question

我的 Heroku RoR 应用程序目前存在问题，导致它无法与 Postgres 数据库通信。在尝试加载我的应用时，我遇到了一般的“出现问题”问题。

运行 heroku logs 会返回以下内容：

2013-03-28T03:46:01+00:00 heroku[web.1]: Stopping all processes with SIGTERM
2013-03-28T03:46:01+00:00 heroku[web.1]: State changed from starting to up
2013-03-28T03:46:01+00:00 app[web.1]: [2013-03-28 03:46:01] ERROR SignalException: SIGTERM
2013-03-28T03:46:01+00:00 app[web.1]:   /usr/local/lib/ruby/1.9.1/webrick/server.rb:90:in `select'
2013-03-28T03:46:05+00:00 heroku[web.1]: Error R12 (Exit timeout) -> At least one process failed to exit within 10 seconds of SIGTERM
2013-03-28T03:46:05+00:00 heroku[web.1]: Stopping remaining processes with SIGKILL
2013-03-28T03:46:07+00:00 heroku[web.1]: Process exited with status 137
2013-03-28T03:48:19+00:00 app[web.1]: Started GET "/projects" for 0.0.0.0 at 2013-03-28 03:48:19 +0000
2013-03-28T03:48:19+00:00 app[web.1]: PG::Error (could not fork new process for connection: Cannot allocate memory
2013-03-28T03:48:19+00:00 app[web.1]: 
2013-03-28T03:48:19+00:00 app[web.1]: could not fork new process for connection: Cannot allocate memory
2013-03-28T03:48:19+00:00 app[web.1]: 
2013-03-28T03:48:19+00:00 app[web.1]: ):
2013-03-28T03:48:19+00:00 app[web.1]:   vendor/bundle/ruby/1.9.1/gems/activerecord-3.2.11/lib/active_record/connection_adapters/postgresql_adapter.rb:1208:in `initialize'
2013-03-28T03:48:19+00:00 app[web.1]: could not fork new process for connection: Cannot allocate memory
2013-03-28T03:48:19+00:00 app[web.1]:   vendor/bundle/ruby/1.9.1/gems/activerecord-3.2.11/lib/active_record/connection_adapters/postgresql_adapter.rb:1208:in `new'
2013-03-28T03:48:19+00:00 app[web.1]:   vendor/bundle/ruby/1.9.1/gems/activerecord-3.2.11/lib/active_record/connection_adapters/postgresql_adapter.rb:326:in `initialize'
2013-03-28T03:48:19+00:00 app[web.1]:   vendor/bundle/ruby/1.9.1/gems/activerecord-3.2.11/lib/active_record/connection_adapters/postgresql_adapter.rb:28:in `new'
2013-03-28T03:48:19+00:00 app[web.1]:   vendor/bundle/ruby/1.9.1/gems/activerecord-3.2.11/lib/active_record/connection_adapters/postgresql_adapter.rb:1208:in `connect'
2013-03-28T03:48:19+00:00 app[web.1]:   vendor/bundle/ruby/1.9.1/gems/activerecord-3.2.11/lib/active_record/connection_adapters/postgresql_adapter.rb:28:in `postgresql_connection'
2013-03-28T03:48:19+00:00 app[web.1]:   vendor/bundle/ruby/1.9.1/gems/activerecord-3.2.11/lib/active_record/connection_adapters/abstract/connection_pool.rb:315:in `new_connection'
2013-03-28T03:48:19+00:00 app[web.1]:   vendor/bundle/ruby/1.9.1/gems/activerecord-3.2.11/lib/active_record/connection_adapters/abstract/connection_pool.rb:325:in `checkout_new_connection'
2013-03-28T03:48:19+00:00 app[web.1]:   vendor/bundle/ruby/1.9.1/gems/activerecord-3.2.11/lib/active_record/connection_adapters/abstract/connection_pool.rb:242:in `loop'
2013-03-28T03:48:19+00:00 app[web.1]:   /usr/local/lib/ruby/1.9.1/monitor.rb:201:in `mon_synchronize'
2013-03-28T03:48:19+00:00 app[web.1]:   vendor/bundle/ruby/1.9.1/gems/activerecord-3.2.11/lib/active_record/connection_adapters/abstract/connection_pool.rb:239:in `checkout'
2013-03-28T03:48:19+00:00 app[web.1]:   vendor/bundle/ruby/1.9.1/gems/activerecord-3.2.11/lib/active_record/connection_adapters/abstract/connection_pool.rb:102:in `block in connection'
2013-03-28T03:48:19+00:00 app[web.1]:   /usr/local/lib/ruby/1.9.1/monitor.rb:201:in `mon_synchronize'
2013-03-28T03:48:19+00:00 app[web.1]:   vendor/bundle/ruby/1.9.1/gems/activerecord-3.2.11/lib/active_record/connection_adapters/abstract/connection_pool.rb:101:in `connection'
2013-03-28T03:48:19+00:00 app[web.1]:   vendor/bundle/ruby/1.9.1/gems/activerecord-3.2.11/lib/active_record/connection_adapters/abstract/connection_pool.rb:247:in `block (2 levels) in checkout'
2013-03-28T03:48:19+00:00 app[web.1]:   vendor/bundle/ruby/1.9.1/gems/activerecord-3.2.11/lib/active_record/connection_adapters/abstract/connection_pool.rb:242:in `block in checkout'
2013-03-28T03:48:19+00:00 app[web.1]:   vendor/bundle/ruby/1.9.1/gems/activerecord-3.2.11/lib/active_record/connection_adapters/abstract/connection_specification.rb:171:in `retrieve_connection'
2013-03-28T03:48:19+00:00 app[web.1]:   vendor/bundle/ruby/1.9.1/gems/activerecord-3.2.11/lib/active_record/connection_adapters/abstract/connection_specification.rb:145:in `connection'
2013-03-28T03:48:19+00:00 app[web.1]:   vendor/bundle/ruby/1.9.1/gems/activerecord-3.2.11/lib/active_record/connection_adapters/abstract/connection_pool.rb:410:in `retrieve_connection'
2013-03-28T03:48:19+00:00 app[web.1]:   vendor/bundle/ruby/1.9.1/gems/activerecord-3.2.11/lib/active_record/query_cache.rb:67:in `rescue in call'
2013-03-28T03:48:19+00:00 app[web.1]:   vendor/bundle/ruby/1.9.1/gems/activerecord-3.2.11/lib/active_record/query_cache.rb:61:in `call'
2013-03-28T03:48:19+00:00 app[web.1]:   vendor/bundle/ruby/1.9.1/gems/actionpack-3.2.11/lib/action_dispatch/middleware/callbacks.rb:28:in `block in call'
2013-03-28T03:48:19+00:00 app[web.1]:   vendor/bundle/ruby/1.9.1/gems/activesupport-3.2.11/lib/active_support/callbacks.rb:405:in `_run__3713961688065884330__call__3295828753336647570__callbacks'
2013-03-28T03:48:19+00:00 app[web.1]:   vendor/bundle/ruby/1.9.1/gems/activerecord-3.2.11/lib/active_record/connection_adapters/abstract/connection_pool.rb:479:in `call'
2013-03-28T03:48:19+00:00 app[web.1]:   vendor/bundle/ruby/1.9.1/gems/activesupport-3.2.11/lib/active_support/callbacks.rb:405:in `__run_callback'
2013-03-28T03:48:19+00:00 app[web.1]:   vendor/bundle/ruby/1.9.1/gems/activesupport-3.2.11/lib/active_support/callbacks.rb:385:in `_run_call_callbacks'
2013-03-28T03:48:19+00:00 app[web.1]:   vendor/bundle/ruby/1.9.1/gems/activesupport-3.2.11/lib/active_support/callbacks.rb:81:in `run_callbacks'
2013-03-28T03:48:19+00:00 app[web.1]:   vendor/bundle/ruby/1.9.1/gems/actionpack-3.2.11/lib/action_dispatch/middleware/show_exceptions.rb:56:in `call'
2013-03-28T03:48:19+00:00 app[web.1]:   vendor/bundle/ruby/1.9.1/gems/actionpack-3.2.11/lib/action_dispatch/middleware/remote_ip.rb:31:in `call'
2013-03-28T03:48:19+00:00 app[web.1]:   vendor/bundle/ruby/1.9.1/gems/actionpack-3.2.11/lib/action_dispatch/middleware/callbacks.rb:27:in `call'
2013-03-28T03:48:19+00:00 app[web.1]:   vendor/bundle/ruby/1.9.1/gems/railties-3.2.11/lib/rails/rack/logger.rb:32:in `call_app'
2013-03-28T03:48:19+00:00 app[web.1]:   vendor/bundle/ruby/1.9.1/gems/actionpack-3.2.11/lib/action_dispatch/middleware/debug_exceptions.rb:16:in `call'
2013-03-28T03:48:19+00:00 app[web.1]:   vendor/bundle/ruby/1.9.1/gems/railties-3.2.11/lib/rails/rack/logger.rb:16:in `block in call'
2013-03-28T03:48:19+00:00 app[web.1]:   vendor/bundle/ruby/1.9.1/gems/activesupport-3.2.11/lib/active_support/tagged_logging.rb:22:in `tagged'
2013-03-28T03:48:19+00:00 app[web.1]:   vendor/bundle/ruby/1.9.1/gems/railties-3.2.11/lib/rails/rack/logger.rb:16:in `call'
2013-03-28T03:48:19+00:00 app[web.1]:   vendor/bundle/ruby/1.9.1/gems/actionpack-3.2.11/lib/action_dispatch/middleware/request_id.rb:22:in `call'
2013-03-28T03:48:19+00:00 app[web.1]:   vendor/bundle/ruby/1.9.1/gems/rack-1.4.5/lib/rack/methodoverride.rb:21:in `call'
2013-03-28T03:48:19+00:00 app[web.1]:   vendor/bundle/ruby/1.9.1/gems/activesupport-3.2.11/lib/active_support/cache/strategy/local_cache.rb:72:in `call'
2013-03-28T03:48:19+00:00 app[web.1]:   vendor/bundle/ruby/1.9.1/gems/rack-1.4.5/lib/rack/runtime.rb:17:in `call'
2013-03-28T03:48:19+00:00 app[web.1]:   vendor/bundle/ruby/1.9.1/gems/rack-1.4.5/lib/rack/lock.rb:15:in `call'
2013-03-28T03:48:19+00:00 app[web.1]:   vendor/bundle/ruby/1.9.1/gems/actionpack-3.2.11/lib/action_dispatch/middleware/static.rb:62:in `call'
2013-03-28T03:48:19+00:00 app[web.1]:   vendor/bundle/ruby/1.9.1/gems/rack-cache-1.2/lib/rack/cache/context.rb:136:in `forward'
2013-03-28T03:48:19+00:00 app[web.1]:   vendor/bundle/ruby/1.9.1/gems/rack-cache-1.2/lib/rack/cache/context.rb:185:in `lookup'
2013-03-28T03:48:19+00:00 app[web.1]:   vendor/bundle/ruby/1.9.1/gems/rack-cache-1.2/lib/rack/cache/context.rb:66:in `call!'
2013-03-28T03:48:19+00:00 app[web.1]:   vendor/bundle/ruby/1.9.1/gems/rack-cache-1.2/lib/rack/cache/context.rb:245:in `fetch'
2013-03-28T03:48:19+00:00 app[web.1]:   vendor/bundle/ruby/1.9.1/gems/rack-cache-1.2/lib/rack/cache/context.rb:51:in `call'
2013-03-28T03:48:19+00:00 app[web.1]:   vendor/bundle/ruby/1.9.1/gems/railties-3.2.11/lib/rails/application.rb:223:in `call'
2013-03-28T03:48:19+00:00 app[web.1]:   vendor/bundle/ruby/1.9.1/gems/railties-3.2.11/lib/rails/engine.rb:479:in `call'
2013-03-28T03:48:19+00:00 app[web.1]:   vendor/bundle/ruby/1.9.1/gems/rack-1.4.5/lib/rack/content_length.rb:14:in `call'
2013-03-28T03:48:19+00:00 app[web.1]:   /usr/local/lib/ruby/1.9.1/webrick/httpserver.rb:70:in `run'
2013-03-28T03:48:19+00:00 app[web.1]:   /usr/local/lib/ruby/1.9.1/webrick/httpserver.rb:111:in `service'
2013-03-28T03:48:19+00:00 app[web.1]:   vendor/bundle/ruby/1.9.1/gems/railties-3.2.11/lib/rails/rack/log_tailer.rb:17:in `call'
2013-03-28T03:48:19+00:00 app[web.1]:   vendor/bundle/ruby/1.9.1/gems/rack-1.4.5/lib/rack/handler/webrick.rb:59:in `service'
2013-03-28T03:48:19+00:00 app[web.1]:   /usr/local/lib/ruby/1.9.1/webrick/server.rb:183:in `block in start_thread'

关键部分在开头：

2013-03-28T03:48:19+00:00 app[web.1]: PG::Error (could not fork new process for connection: Cannot allocate memory
2013-03-28T03:48:19+00:00 app[web.1]: 
2013-03-28T03:48:19+00:00 app[web.1]: could not fork new process for connection: Cannot allocate memory
2013-03-28T03:48:19+00:00 app[web.1]: 
2013-03-28T03:48:19+00:00 app[web.1]: ):

我发现this 的问题，我试图限制退货的数量，但这似乎不是问题。我的数据库中只有两个项目，它们都有以下模型：

class Project < ActiveRecord::Base
  attr_accessible :description, :image, :name, :url
end

哪些是文本/字符串字段。

Answer 1

我注意到您在大约 15 分钟前发布了此消息。在过去的半个小时里，我也开始在 heroku 上看到这个错误（连接到 dev/free postgres db）。我想知道这是不是他们的问题？他们的状态页面没有给出任何提示。

Answer 2

你知道，这个错误的关键部分是：

could not fork new process for connection: Cannot allocate memory

这表明主机、容器或 ulimit 上下文没有更多可用内存。创建一个新进程需要为堆栈和其他各种基本数据结构分配内存。它的内存不多，因此无法派生进程表明您的 RAM 完全不足。

在普通主机上，您将登录（如果可以的话）、检查正在运行的进程、杀死有问题的进程、为有问题的服务设置内存限制等。这在 Heroku 上并不是一个真正的选项，但您 可以：

• 减少您正在使用的并发连接数；和
• 减少每个连接的 work_mem 和 maintenance_work_mem

如果您超出了容器中可用的 RAM，这将有所帮助。如果整个主机内存不足，它对你没有任何好处。

我为一位在升级更大的 Heroku 计划后遇到类似问题的客户提供了一些紧急支持。对于发生的事情，我们从来没有得到可靠的答案，但客户最终 - 在几次尝试和几天之后 - 创建了他们数据库的追随者（副本），然后故障转移到它。这解决了问题，表明问题可能与托管该数据库实例的容器有关。如果您可以成功创建关注者，请尝试此操作。您可能必须减少连接数和work_mem 等才能做到这一点。

一般来说，请考虑 PostgreSQL 在活动连接数量相对较少且连接池良好的情况下表现最佳。使用 20 或 30 个池连接通常会比使用 200 或 300 且没有池器时获得更好的吞吐量。减少连接数，直到问题消失，您也可能会看到性能提升。

（披露：我在 2ndQuadrant 工作，这是另一家 PostgreSQL 支持和服务公司）。