Resque 队列：没有活动线程？答案

【问题标题】：Resque queue: No live threads?Resque 队列：没有活动线程？
【发布时间】：2015-10-10 10:11:10
【问题描述】：

我一直在尝试运行一个 resque 任务。 Resque 似乎正在排队和运行任务；然而，任务似乎经常在执行时“陷入僵局”。

示例错误如下：（取自 Resque::Failure 输出）

 {"failed_at"=>"2015/07/20 22:34:46 UTC",
  "payload"=>{"class"=>"ImportHawaiiEventsOnline", "args"=>[]},
  "exception"=>"fatal",
  "error"=>"No live threads left. Deadlock?",
"backtrace"=>
   ["/usr/local/lib/ruby/2.1.0/open3.rb:262:in `value'",
    "/usr/local/lib/ruby/2.1.0/open3.rb:262:in `block in capture3'",
    "/usr/local/lib/ruby/2.1.0/open3.rb:199:in `popen_run'",
    "/usr/local/lib/ruby/2.1.0/open3.rb:93:in `popen3'",
    "/usr/local/lib/ruby/2.1.0/open3.rb:252:in `capture3'",
    "/srv/releases/4/vendor/bundle/ruby/2.1.0/gems/mini_magick-4.2.7/lib/mini_magick/shell.rb:47:in `execute_open3'",
    "/srv/releases/4/vendor/bundle/ruby/2.1.0/gems/mini_magick-4.2.7/lib/mini_magick/shell.rb:36:in `block (2 levels) in execute'",
    "/usr/local/lib/ruby/2.1.0/timeout.rb:76:in `timeout'",
    "/srv/releases/4/vendor/bundle/ruby/2.1.0/gems/mini_magick-4.2.7/lib/mini_magick/shell.rb:35:in `block in execute'",
    "/srv/releases/4/vendor/bundle/ruby/2.1.0/gems/mini_magick-4.2.7/lib/mini_magick/logger.rb:32:in `call'",
    "/srv/releases/4/vendor/bundle/ruby/2.1.0/gems/mini_magick-4.2.7/lib/mini_magick/logger.rb:32:in `block in benchmark'",
    "/usr/local/lib/ruby/2.1.0/benchmark.rb:294:in `realtime'",
    "/srv/releases/4/vendor/bundle/ruby/2.1.0/gems/mini_magick-4.2.7/lib/mini_magick/logger.rb:32:in `benchmark'",
    "/srv/releases/4/vendor/bundle/ruby/2.1.0/gems/mini_magick-4.2.7/lib/mini_magick/logger.rb:21:in `debug'",
    "/srv/releases/4/vendor/bundle/ruby/2.1.0/gems/mini_magick-4.2.7/lib/mini_magick/shell.rb:34:in `execute'",
    "/srv/releases/4/vendor/bundle/ruby/2.1.0/gems/mini_magick-4.2.7/lib/mini_magick/shell.rb:18:in `run'",
    "/srv/releases/4/vendor/bundle/ruby/2.1.0/gems/mini_magick-4.2.7/lib/mini_magick/tool.rb:92:in `call'",
    "/srv/releases/4/vendor/bundle/ruby/2.1.0/gems/mini_magick-4.2.7/lib/mini_magick/tool.rb:53:in `new'",
    "/srv/releases/4/vendor/bundle/ruby/2.1.0/gems/mini_magick-4.2.7/lib/mini_magick/image.rb:477:in `identify'",
    "/srv/releases/4/vendor/bundle/ruby/2.1.0/gems/mini_magick-4.2.7/lib/mini_magick/image.rb:198:in `validate!'",
    "/srv/releases/4/vendor/bundle/ruby/2.1.0/gems/mini_magick-4.2.7/lib/mini_magick/image.rb:113:in `block in create'",
    "/srv/releases/4/vendor/bundle/ruby/2.1.0/gems/mini_magick-4.2.7/lib/mini_magick/image.rb:112:in `tap'",
    "/srv/releases/4/vendor/bundle/ruby/2.1.0/gems/mini_magick-4.2.7/lib/mini_magick/image.rb:112:in `create'",
    "/srv/releases/4/vendor/bundle/ruby/2.1.0/gems/mini_magick-4.2.7/lib/mini_magick/image.rb:34:in `read'",
    "/srv/releases/4/vendor/bundle/ruby/2.1.0/gems/mini_magick-4.2.7/lib/mini_magick/image.rb:90:in `block in open'",
    "/usr/local/lib/ruby/2.1.0/open-uri.rb:36:in `open'",
    "/usr/local/lib/ruby/2.1.0/open-uri.rb:36:in `open'",
    "/srv/releases/4/vendor/bundle/ruby/2.1.0/gems/mini_magick-4.2.7/lib/mini_magick/image.rb:89:in `open'",
    "/srv/releases/4/app/uploaders/photo_uploader.rb:57:in `get_geometry'",
    "/srv/releases/4/vendor/bundle/ruby/2.1.0/gems/carrierwave-0.10.0/lib/carrierwave/uploader/processing.rb:84:in `block in process!'",
    "/srv/releases/4/vendor/bundle/ruby/2.1.0/gems/carrierwave-0.10.0/lib/carrierwave/uploader/processing.rb:76:in `each'",
    "/srv/releases/4/vendor/bundle/ruby/2.1.0/gems/carrierwave-0.10.0/lib/carrierwave/uploader/processing.rb:76:in `process!'",
    "/srv/releases/4/vendor/bundle/ruby/2.1.0/gems/carrierwave-0.10.0/lib/carrierwave/uploader/callbacks.rb:18:in `block in with_callbacks'",
    "/srv/releases/4/vendor/bundle/ruby/2.1.0/gems/carrierwave-0.10.0/lib/carrierwave/uploader/callbacks.rb:18:in `each'",
    "/srv/releases/4/vendor/bundle/ruby/2.1.0/gems/carrierwave-0.10.0/lib/carrierwave/uploader/callbacks.rb:18:in `with_callbacks'",
    "/srv/releases/4/vendor/bundle/ruby/2.1.0/gems/carrierwave-0.10.0/lib/carrierwave/uploader/cache.rb:122:in `cache!'",
    "/srv/releases/4/vendor/bundle/ruby/2.1.0/gems/carrierwave-0.10.0/lib/carrierwave/uploader/download.rb:72:in `download!'",
    "/srv/releases/4/vendor/bundle/ruby/2.1.0/gems/carrierwave-0.10.0/lib/carrierwave/mount.rb:356:in `remote_url='",
    "/srv/releases/4/vendor/bundle/ruby/2.1.0/gems/carrierwave-0.10.0/lib/carrierwave/mount.rb:187:in `remote_photo_url='",
    "/srv/releases/4/vendor/bundle/ruby/2.1.0/gems/carrierwave-0.10.0/lib/carrierwave/orm/activerecord.rb:45:in `remote_photo_url='",
    "/srv/releases/4/vendor/bundle/ruby/2.1.0/gems/activerecord-4.1.1/lib/active_record/attribute_assignment.rb:45:in `public_send'",
    "/srv/releases/4/vendor/bundle/ruby/2.1.0/gems/activerecord-4.1.1/lib/active_record/attribute_assignment.rb:45:in `_assign_attribute'",
    "/srv/releases/4/vendor/bundle/ruby/2.1.0/gems/activerecord-4.1.1/lib/active_record/attribute_assignment.rb:32:in `block in assign_attributes'",
    "/srv/releases/4/vendor/bundle/ruby/2.1.0/gems/activerecord-4.1.1/lib/active_record/attribute_assignment.rb:26:in `each'",
    "/srv/releases/4/vendor/bundle/ruby/2.1.0/gems/activerecord-4.1.1/lib/active_record/attribute_assignment.rb:26:in `assign_attributes'",
    "/srv/releases/4/event_jobs/generic_import.rb:45:in `event_endpoint'",
    "/srv/releases/4/event_jobs/generic_import.rb:113:in `save_event'",
    "/srv/releases/4/event_jobs/hawaiieventonline/import.rb:26:in `block (2 levels) in get_parser'",
    "/srv/releases/4/vendor/bundle/ruby/2.1.0/gems/nokogiri-1.6.6.2/lib/nokogiri/xml/node_set.rb:187:in `block in each'",
    "/srv/releases/4/vendor/bundle/ruby/2.1.0/gems/nokogiri-1.6.6.2/lib/nokogiri/xml/node_set.rb:186:in `upto'",
    "/srv/releases/4/vendor/bundle/ruby/2.1.0/gems/nokogiri-1.6.6.2/lib/nokogiri/xml/node_set.rb:186:in `each'",
    "/srv/releases/4/event_jobs/hawaiieventonline/import.rb:24:in `each_with_index'",
    "/srv/releases/4/event_jobs/hawaiieventonline/import.rb:24:in `block in     get_parser'",
    "/srv/releases/4/vendor/bundle/ruby/2.1.0/gems/nokogiri-1.6.6.2/lib/nokogiri/xml/node_set.rb:187:in `block in each'",
    "/srv/releases/4/vendor/bundle/ruby/2.1.0/gems/nokogiri-1.6.6.2/lib/nokogiri/xml/node_set.rb:186:in `upto'",
    "/srv/releases/4/vendor/bundle/ruby/2.1.0/gems/nokogiri-1.6.6.2/lib/nokogiri/xml/node_set.rb:186:in `each'",
    "/srv/releases/4/event_jobs/hawaiieventonline/import.rb:21:in `get_parser'",
    "/srv/releases/4/event_jobs/generic_import.rb:22:in `run_parse'",
    "/srv/releases/4/app/jobs/import_hawaiieventsonline.rb:17:in `perform'"],
  "worker"=>
   "be920dfe9b17:5637:`(queues)`",
  "queue"=>"`(queue)`"},

我猜测它与 mini_magick gem 相关——尽管它暗示死锁似乎很奇怪，因为我只运行一个线程。我猜这可能是由于异步发出的大量请求 - 尽管我不完全了解 mini_magick 和carrierwave gem。

有人对在哪里解决这个线程问题有任何建议吗？我目前正在运行 17 个队列，其中一个工作线程处理所有这些队列，而且它似乎运行良好 - 只是保存到云中可能存在问题。尝试搜索与 mini_magick/carrierwave 相关的任何内容，但线程问题似乎没有显示任何内容。

提前致谢。

【问题讨论】：

我自己也遇到了一个非常相似的错误 - 你找到了根本原因吗？

标签： ruby-on-rails multithreading carrierwave resque minimagick

【解决方案1】：

我很好奇，您是否在代码中运行 Resque-Pool 或任何其他分叉/线程？ MiniMagick 使用 open3 来运行 shell 命令，它显然有一个reputation，因为它不能很好地处理线程。幸运的是，MiniMagick 允许您将 open3 替换为 Posix-Spawn。

我在使用线程工作者（Resque-Pool 作业）时遇到了同样的错误，切换到 Posix-Spawn 似乎已经为我解决了这个错误。

【讨论】：