【问题标题】:Why Storm is not replaying failed message on Cluster at Work but replaying on cluster mode at local desktop为什么 Storm 不在工作集群上重播失败消息,而是在本地桌面以集群模式重播
【发布时间】:2020-04-21 01:09:03
【问题描述】:

这是我要执行的代码。我故意在螺栓上失败了。这样我就可以看到失败的消息被风暴重播。但看起来这并没有发生。

public static class FastRandomSentenceSpout extends BaseRichSpout {
  SpoutOutputCollector _collector;
  Random _rand;
   private static final String[] CHOICES = {
       "marry had a little lamb whos fleese was white as snow",
       "and every where that marry went the lamb was sure to go"
   };

   @Override
   public void open(Map conf, TopologyContext context, SpoutOutputCollector collector) {
      _collector = collector;
      _rand = ThreadLocalRandom.current();
    }

   @Override
   public void nextTuple() {
      String sentence = CHOICES[_rand.nextInt(CHOICES.length)];
      _collector.emit(new Values(sentence), sentence);
   }

   @Override
   public void fail(Object id) {
      System.out.println("RAVI: the failedObjectId = "+id);
      _collector.emit(new Values(id), id);
   }

   @Override
   public void declareOutputFields(OutputFieldsDeclarer declarer) {
      declarer.declare(new Fields("sentence"));
   }
 }

这里是关于Split sentence Bolt的详细信息。我故意失败的地方。

 public static class SplitSentence extends BaseRichBolt 
 {
     OutputCollector _collector;
     @Override
     public void prepare(Map conf,
                     TopologyContext context,
                     OutputCollector collector)
    {
       _collector = collector;
    }

这是发生故障的函数

    @Override
    public void execute(Tuple tuple) 
    {
        String sentence = tuple.getString(0);
        System.out.println("sentence = "+sentence);
        if(sentence.equals("marry had a little lamb whos fleese was white as snow"))
        {
           System.out.println("going to fail");
           _collector.fail(tuple);
        }
        else
        { 
           for (String word: sentence.split("\\s+")) {
              _collector.emit(tuple, new Values(word, 1));
           }
           _collector.ack(tuple);
        }   
     }

     @Override
     public void declareOutputFields(OutputFieldsDeclarer declarer) {
         declarer.declare(new Fields("word", "count"));
     }
  }

这是驱动代码的详细信息。 public static void main(String[] args) 抛出异常 {

   TopologyBuilder builder = new TopologyBuilder();

   builder.setSpout("spout", new FastRandomSentenceSpout(), 4);

   builder.setBolt("split", new SplitSentence(), 4).shuffleGrouping("spout");


   Config conf = new Config();
   conf.registerMetricsConsumer(
             org.apache.storm.metric.LoggingMetricsConsumer.class);


   String name = "wc-test";
   if (args != null && args.length > 0) {
       name = args[0];
   }

   conf.setNumWorkers(1);
   StormSubmitter.submitTopologyWithProgressBar(name, 
                                                conf,
                                                builder.createTopology());

  }

【问题讨论】:

  • 是否有覆盖 conf.setNumAckers() 的全局集群级别设置?我尝试明确设置 conf.setNumAckers(1);但结果还是一样。

标签: java apache-storm


【解决方案1】:

原来是因为storm.yaml中提到的全局设置。具体设置是

topology.acker.executors: 0

【讨论】:

    猜你喜欢
    • 1970-01-01
    • 1970-01-01
    • 2013-07-21
    • 1970-01-01
    • 2019-05-08
    • 2020-01-14
    • 1970-01-01
    • 1970-01-01
    • 2020-12-09
    相关资源
    最近更新 更多