NoMemoryError：分配内存失败 - Ruby on Rails

Question

我生成了一个在其他场合对我有用的简单脚本，但这是因为循环中的信息量会产生 NoMemoryError 错误。

我有 16 GB 的内存和大量可用的虚拟内存。当我执行测试时，RAM 内存已完全填满。

脚本是：

  require 'rest-client'
  require 'json'
  require 'open-uri'
  require 'csv'

  def self.qos7705egressdiscard_summary
   xml = File.read("#{Rails.root}/public/Discard_qos7705egress.xml")
   data = RestClient.post("http://10.140.255.1:8080/xmlapi/invoke", xml,{:"Content-Type" => 'application/soap+xml'})
   data_parsed = Hash.from_xml(data)
   return data_parsed
  end

  def self.samqos7705egressdiscardtotal_table
   tabletotal = Hash.new
   data_stats = qos7705egressdiscard_summary['Envelope']['Body']['findResponse']['result']['service.SapEgrQosQueueStatsLogRecord']

   data_stats.map do |qosdiscard_device|
    @devicetotal = qosdiscard_device["monitoredObjectSiteName"]
    @servicetotal = qosdiscard_device["monitoredObjectPointer"]
    @porttotal = qosdiscard_device["displayedName"]
    @queueIdtotal = qosdiscard_device["queueId"]
    @discardinproftotal = qosdiscard_device["droppedInProfOctets"].to_i
    @discardoutproftotal = qosdiscard_device["droppedOutProfOctets"].to_i
    time_unixtotal = (qosdiscard_device["timeCaptured"]).to_i/1000
    @timeCapturedtotal = Time.at(time_unixtotal).strftime("%B %e, %Y at %I:%M %p")
    @discardtotal = @discardinproftotal + @discardoutproftotal
    @device_int_stats_total = (@devicetotal+@porttotal+@queueIdtotal).to_s
     hash = Hash[devicetotal: @devicetotal, servicetotal: @servicetotal, porttotal: @porttotal, queueIdtotal: @queueIdtotal, discardtotal: @discardtotal, device_int_stats_total: @device_int_stats_total, timeCapturedtotal: @timeCapturedtotal, time_unixtotal: time_unixtotal]
     tabletotal << hash
     #tabletotal.write(hash)
   end
  end

确切的错误是：

NoMemoryError: failed to allocate memory
        from /usr/local/lib/ruby/2.2.0/irb/inspector.rb:108:in `inspect'
        from /usr/local/lib/ruby/2.2.0/irb/inspector.rb:108:in `block in <module:IRB>'
        from /usr/local/lib/ruby/2.2.0/irb/inspector.rb:101:in `call'
        from /usr/local/lib/ruby/2.2.0/irb/inspector.rb:101:in `inspect_value'
        from /usr/local/lib/ruby/2.2.0/irb/context.rb:383:in `inspect_last_value'
        from /usr/local/lib/ruby/2.2.0/irb.rb:661:in `output_value'
        from /usr/local/lib/ruby/2.2.0/irb.rb:490:in `block (2 levels) in eval_input'
        from /usr/local/lib/ruby/2.2.0/irb.rb:623:in `signal_status'
        from /usr/local/lib/ruby/2.2.0/irb.rb:486:in `block in eval_input'
        from /usr/local/lib/ruby/2.2.0/irb/ruby-lex.rb:245:in `block (2 levels) in each_top_level_statement'
        from /usr/local/lib/ruby/2.2.0/irb/ruby-lex.rb:231:in `loop'
        from /usr/local/lib/ruby/2.2.0/irb/ruby-lex.rb:231:in `block in each_top_level_statement'
        from /usr/local/lib/ruby/2.2.0/irb/ruby-lex.rb:230:in `catch'
        from /usr/local/lib/ruby/2.2.0/irb/ruby-lex.rb:230:in `each_top_level_statement'
        from /usr/local/lib/ruby/2.2.0/irb.rb:485:in `eval_input'
        from /usr/local/lib/ruby/2.2.0/irb.rb:395:in `block in start'
        from /usr/local/lib/ruby/2.2.0/irb.rb:394:in `catch'
        from /usr/local/lib/ruby/2.2.0/irb.rb:394:in `start'
        from /usr/local/lib/ruby/gems/2.2.0/gems/railties-4.2.0/lib/rails/commands/console.rb:110:in `start'
        from /usr/local/lib/ruby/gems/2.2.0/gems/railties-4.2.0/lib/rails/commands/console.rb:9:in `start'
        from /usr/local/lib/ruby/gems/2.2.0/gems/railties-4.2.0/lib/rails/commands/commands_tasks.rb:68:in `console'
        from /usr/local/lib/ruby/gems/2.2.0/gems/railties-4.2.0/lib/rails/commands/commands_tasks.rb:39:in `run_command!'
        from /usr/local/lib/ruby/gems/2.2.0/gems/railties-4.2.0/lib/rails/commands.rb:17:in `<top (required)>'
        from bin/rails:4:in `require'
        from bin/rails:4:in `<main>'Maybe IRB bug!

在第 25 行，我添加了这个 tabletotal.write (hash) 以便能够将其写入磁盘而不是内存，但出现以下错误：

NoMethodError: undefined method `write' for {}:Hash

这里有什么问题？还有怎么解决？

Answer 1

从减少 cr*p 开始：

def extract_data(input)
  {
    devicetotal: input["monitoredObjectSiteName"],
    servicetotal: input["monitoredObjectPointer"],
    porttotal: input["displayedName"],
    queueIdtotal: input["queueId"],
    discardinproftotal:  input["droppedInProfOctets"].to_i,
    discardoutproftotal: input["droppedOutProfOctets"].to_i,
    time_unixtotal: input["timeCaptured"].to_i/1000
  }.tap do |h|
    h[:timeCapturedtotal] = Time.at(h[:time_unixtotal]).strftime("%B %e, %Y at %I:%M %p"),
    h[:discardtotal] = h[:discardinproftotal] + h[:discardoutproftotal]
    h[:device_int_stats_total] =(h[:devicetotal]+h[:porttotal]+h[:queueIdtotal]).to_s
  end
end

这种方法非常容易测试，因为您只需插入输入并编写关于输出的断言。

如果您想映射并将其应用于输入数组，您可以这样做：

data_stats.map(&:extract_data)

您的代码尝试输出散列，但使用铲操作符，就像在数组上一样。您需要确定适当的输出是数组还是散列。

我需要所有这些变量，因为我在表格中的 html 中使用它们 格式。

这不起作用 - 实例变量将仅包含最后一个元素的值，因为它们在每次迭代中都会被覆盖。

您需要遍历视图中的哈希或对象数组：

<% @stats.each do |s| %>
  <tr>
    <td><%= s[:devicetotal] %></td>
    # ...
  </tr>
<% end %>

如果你真的想使用实例变量，你需要为每个元素创建一个对象实例：

class DataStat
  include ActiveModel::AttributeAssignment
  attr_accessor :devicetotal, :servicetotal # ...

  def self.from_import(input)
    self.new(
      devicetotal: input["monitoredObjectSiteName"],
      servicetotal: input["monitoredObjectPointer"],
      # ...
    )
  end  
end

@stats = data_stats.map { |ds| DataStat.from_import(ds) }

您还需要处理内存不足的问题，因为您只是草率地将整个 XML 文档转换为 Ruby 哈希。您需要改为parse it with Nokogiri 并提取您实际需要的内容。