【问题标题】:Verifying Storm topology processing and Storm UI generating 404 error验证 Storm 拓扑处理和 Storm UI 生成 404 错误
【发布时间】:2014-04-27 23:21:08
【问题描述】:

我已经使用多个来源设置了一个 Storm 开发集群,例如 Storm Real-time Processing Cookbook、P. Taylor Goetz 在 OSX 上设置 Storm,当然还有来自 Nathan Marz 的所有内容.目前,我使用 Puppet 和 Vagrant 在一台机器上设置了一个集群,该机器具有用于 Nimbus、Zookeeper 和两个 Supervisor 的 VM。本地机器本身就是主人。我似乎可以从storm-starter https://github.com/nathanmarz/storm-starter 提交示例拓扑并让它们运行,但我在通过Storm 运行各种工具或验证集群是否实际工作时遇到了一些问题!

问题: 每当我从它的 bin 中运行任何 Storm 工具时,例如storm ui 或storm logviewer,控制台都会立即吐出以下内容并且通常会挂起:

Running: java -server -Dstorm.options= -Dstorm.home=/Users/jhogan/Development/storm -Djava.library.path=/usr/local/lib:/opt/local/lib:/usr/lib -Dstorm.conf.file= -cp /Users/jhogan/Development/storm/lib/asm-4.0.jar:/Users/jhogan/Development/storm/lib/carbonite-1.3.2.jar:/Users/jhogan/Development/storm/lib/clj-stacktrace-0.2.4.jar:/Users/jhogan/Development/storm/lib/clj-time-0.4.1.jar:/Users/jhogan/Development/storm/lib/clojure-1.4.0.jar:/Users/jhogan/Development/storm/lib/clout-1.0.1.jar:/Users/jhogan/Development/storm/lib/commons-codec-1.4.jar:/Users/jhogan/Development/storm/lib/commons-exec-1.1.jar:/Users/jhogan/Development/storm/lib/commons-fileupload-1.2.1.jar:/Users/jhogan/Development/storm/lib/commons-io-1.4.jar:/Users/jhogan/Development/storm/lib/commons-lang-2.5.jar:/Users/jhogan/Development/storm/lib/commons-logging-1.1.1.jar:/Users/jhogan/Development/storm/lib/compojure-1.1.3.jar:/Users/jhogan/Development/storm/lib/core.incubator-0.1.0.jar:/Users/jhogan/Development/storm/lib/curator-client-1.0.1.jar:/Users/jhogan/Development/storm/lib/curator-framework-1.0.1.jar:/Users/jhogan/Development/storm/lib/disruptor-2.10.1.jar:/Users/jhogan/Development/storm/lib/guava-13.0.jar:/Users/jhogan/Development/storm/lib/hiccup-0.3.6.jar:/Users/jhogan/Development/storm/lib/httpclient-4.1.1.jar:/Users/jhogan/Development/storm/lib/httpcore-4.1.jar:/Users/jhogan/Development/storm/lib/jetty-6.1.26.jar:/Users/jhogan/Development/storm/lib/jetty-util-6.1.26.jar:/Users/jhogan/Development/storm/lib/jgrapht-core-0.9.0.jar:/Users/jhogan/Development/storm/lib/jline-2.11.jar:/Users/jhogan/Development/storm/lib/joda-time-2.0.jar:/Users/jhogan/Development/storm/lib/json-simple-1.1.jar:/Users/jhogan/Development/storm/lib/junit-3.8.1.jar:/Users/jhogan/Development/storm/lib/kryo-2.17.jar:/Users/jhogan/Development/storm/lib/log4j-over-slf4j-1.6.6.jar:/Users/jhogan/Development/storm/lib/logback-classic-1.0.6.jar:/Users/jhogan/Development/storm/lib/logback-core-1.0.6.jar:/Users/jhogan/Development/storm/lib/math.numeric-tower-0.0.1.jar:/Users/jhogan/Development/storm/lib/meat-locker-0.3.1.jar:/Users/jhogan/Development/storm/lib/minlog-1.2.jar:/Users/jhogan/Development/storm/lib/netty-3.6.3.Final.jar:/Users/jhogan/Development/storm/lib/objenesis-1.2.jar:/Users/jhogan/Development/storm/lib/reflectasm-1.07-shaded.jar:/Users/jhogan/Development/storm/lib/ring-core-1.1.5.jar:/Users/jhogan/Development/storm/lib/ring-devel-0.3.11.jar:/Users/jhogan/Development/storm/lib/ring-jetty-adapter-0.3.11.jar:/Users/jhogan/Development/storm/lib/ring-servlet-0.3.11.jar:/Users/jhogan/Development/storm/lib/servlet-api-2.5-20081211.jar:/Users/jhogan/Development/storm/lib/servlet-api-2.5.jar:/Users/jhogan/Development/storm/lib/slf4j-api-1.6.5.jar:/Users/jhogan/Development/storm/lib/snakeyaml-1.11.jar:/Users/jhogan/Development/storm/lib/storm-core-0.9.1-incubating.jar:/Users/jhogan/Development/storm/lib/tools.cli-0.2.2.jar:/Users/jhogan/Development/storm/lib/tools.logging-0.2.3.jar:/Users/jhogan/Development/storm/lib/tools.macro-0.1.0.jar:/Users/jhogan/Development/storm/lib/zookeeper-3.3.3.jar:/Users/jhogan/Development/storm:/Users/jhogan/Development/storm/conf -Xmx768m -Dlogfile.name=ui.log -Dlogback.configurationFile=/Users/jhogan/Development/storm/logback/cluster.xml backtype.storm.ui.core

在这种特殊情况下,我想启动storm ui。 ui登录storm(从master机器):

2014-03-20 12:26:24 o.m.log [INFO] Logging to Logger[org.mortbay.log] via org.mortbay.log.Slf4jLog 2014-03-20 12:26:24 o.m.log [INFO] jetty-6.1.26 2014-03-20 12:26:24 o.m.log [INFO] Started SocketConnector@0.0.0.0:8080

.. 没有什么意义,因为我假设我需要连接到从 Nimbus 机器生成的 UI:

流浪文件: boxes = [ { :name => :nimbus, :ip => '10.200.1.100', :cpus =>2, :memory => 512, :instance => 'm1.small' }, { :name => :supervisor1, :ip => '10.200.1.101', :cpus =>4, :memory => 1024, :instance => 'm1.medium' }, { :name => :supervisor2, :ip => '10.200.1.102', :cpus =>4, :memory => 1024, :instance => 'm1.medium' }, { :name => :zookeeper1, :ip => '10.200.1.201', :cpus =>1, :memory => 1024, :instance => 'm1.small' }, ]

在任何一种情况下,我都会收到页面的 404 错误。

运行“风暴名单”:

`1941 [main] INFO  backtype.storm.thrift - Connecting to Nimbus at 10.200.1.100:6627
Exception in thread "main" org.apache.thrift7.transport.TTransportException: java.net.ConnectException: Connection refused
    at org.apache.thrift7.transport.TSocket.open(TSocket.java:183)
    at org.apache.thrift7.transport.TFramedTransport.open(TFramedTransport.java:81)
    at backtype.storm.thrift$nimbus_client_and_conn.invoke(thrift.clj:71)
    at backtype.storm.command.list$_main.invoke(list.clj:22)
    at clojure.lang.AFn.applyToHelper(AFn.java:159)
    at clojure.lang.AFn.applyTo(AFn.java:151)
    at backtype.storm.command.list.main(Unknown Source)
Caused by: java.net.ConnectException: Connection refused
    at java.net.PlainSocketImpl.socketConnect(Native Method)
    at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339)
    at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200)
    at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182)
    at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392)
    at java.net.Socket.connect(Socket.java:579)
    at org.apache.thrift7.transport.TSocket.open(TSocket.java:178)
    ... 6 more`

每台机器似乎都已正确设置和运行,我可以使用 Vagrant ssh ssh 进入其中的任何一台。我可以提交拓扑:

`storm jar storm-starter-0.0.1-SNAPSHOT-jar-with-dependencies.jar storm.starter.WordCountTopology`

并让它们看似运行,但我不确定如何检查系统是否实际运行正确。在哪里查看拓扑进度的日志?为什么我无法连接到文档中指定的 10.200.1.100:8080 上的storm ui?

我对这个系统很陌生,所以如果我没有发布正确的数据,我深表歉意。让我知道我需要发布 Puppet、Vagrant 或 Storm 的哪些部分来协助解决此问题。

谢谢!

【问题讨论】:

    标签: vagrant puppet apache-storm


    【解决方案1】:

    在我看来,您可能有端口问题。确保 nimbus、zookeeper 和监督者都能够通过适当的端口相互交谈。

    至于日志,您可以查看每个主管的工作人员日志以了解实际发生的情况。

    【讨论】:

      猜你喜欢
      • 1970-01-01
      • 1970-01-01
      • 2016-08-30
      • 1970-01-01
      • 2016-04-16
      • 2019-05-24
      • 1970-01-01
      • 1970-01-01
      • 1970-01-01
      相关资源
      最近更新 更多