ElasticSearch JarHell com.esotericsoftware.minlog.Log$Logger

Question

我需要在 Storm 中使用 node-client 编写一个弹性搜索 Bolt。

我不想使用 Transport-client，因为它的效率较低（涉及节点之间的额外一跳）。

所以我尝试使用以下代码构建自己的 ES 螺栓：

public void prepare(Map stormConf, TopologyContext context, OutputCollector collector) {

  Settings settings = Settings.settingsBuilder()
    .put("http.enabled", false)
    .put("discovery.zen.ping.multicast.enabled", false)
    .put("discovery.zen.ping.unicast.hosts", hosts)
    .put("discovery.zen.minimum_master_nodes", 1)
    .put("path.home", pathHome)
    .build();

  client = NodeBuilder.nodeBuilder()
    .settings(settings)
    .clusterName(esCluster)
    .data(false)
    .client(true)
    .node().client();
}

我的 elastic-search-bolt 中的上述方法给出了以下异常：

java.lang.IllegalStateException: failed to load bundle [file:/etc/elasticsearch/elasticsearch-2.2.0/modules/lang-expression/antlr4-runtime-4.5.1-1.jar, file:/etc/elasticsearch/elasticsearch-2.2.0/modules/lang-expression/asm-5.0.4.jar, file:/etc/elasticsearch/elasticsearch-2.2.0/modules/lang-expression/asm-commons-5.0.4.jar, file:/etc/elasticsearch/elasticsearch-2.2.0/modules/lang-expression/lang-expression-2.2.0.jar, file:/etc/elasticsearch/elasticsearch-2.2.0/modules/lang-expression/lucene-expressions-5.4.1.jar] due to jar hell

    at org.elasticsearch.plugins.PluginsService.loadBundles(PluginsService.java:421) ~[storm_es_bolt-0.0.1-SNAPSHOT.jar:?]
    at org.elasticsearch.plugins.PluginsService.<init>(PluginsService.java:115) ~[storm_es_bolt-0.0.1-SNAPSHOT.jar:?]
    at org.elasticsearch.node.Node.<init>(Node.java:146) ~[storm_es_bolt-0.0.1-SNAPSHOT.jar:?]
    at org.elasticsearch.node.Node.<init>(Node.java:128) ~[storm_es_bolt-0.0.1-SNAPSHOT.jar:?]
    at org.elasticsearch.node.NodeBuilder.build(NodeBuilder.java:145) ~[storm_es_bolt-0.0.1-SNAPSHOT.jar:?]
    at org.elasticsearch.node.NodeBuilder.node(NodeBuilder.java:152) ~[storm_es_bolt-0.0.1-SNAPSHOT.jar:?]
    at mypkg.storm.NodeClientBolt.prepare(NodeClientBolt.java:89) ~[storm_es_bolt-0.0.1-SNAPSHOT.jar:?]
    at backtype.storm.daemon.executor$fn__5694$fn__5707.invoke(executor.clj:757) ~[storm-core-0.10.0.jar:0.10.0]
    at backtype.storm.util$async_loop$fn__545.invoke(util.clj:477) [storm-core-0.10.0.jar:0.10.0]
    at clojure.lang.AFn.run(AFn.java:22) [clojure-1.6.0.jar:?]
    at java.lang.Thread.run(Thread.java:745) [?:1.8.0_71]

Caused by: java.lang.IllegalStateException: jar hell!

class: com.esotericsoftware.minlog.Log$Logger
jar1: /etc/storm/storm-0.10.0.nimbus/lib/kryo-2.21.jar
jar2: /etc/storm/storm-0.10.0.nimbus/lib/minlog-1.2.jar

    at org.elasticsearch.bootstrap.JarHell.checkClass(JarHell.java:280) ~[storm_es_bolt-0.0.1-SNAPSHOT.jar:?]
    at org.elasticsearch.bootstrap.JarHell.checkJarHell(JarHell.java:186) ~[storm_es_bolt-0.0.1-SNAPSHOT.jar:?]
    at org.elasticsearch.plugins.PluginsService.loadBundles(PluginsService.java:419) ~[storm_es_bolt-0.0.1-SNAPSHOT.jar:?]
    ... 10 more

编辑： Jeremie的好建议解决了上述问题，但后来我遇到了另一个问题（非常相似）：

Caused by: java.lang.IllegalStateException: jar hell!
class: org.objectweb.asm.AnnotationVisitor
jar1: /etc/storm/storm-0.10.0.nimbus/lib/asm-4.0.jar
jar2: /etc/elasticsearch/elasticsearch-2.2.0/modules/lang-expression/asm-5.0.4.jar
at org.elasticsearch.bootstrap.JarHell.checkClass(JarHell.java:280) ~[storm_hello_world-0.0.1-SNAPSHOT.jar:?]
at org.elasticsearch.bootstrap.JarHell.checkJarHell(JarHell.java:186) ~[storm_hello_world-0.0.1-SNAPSHOT.jar:?]
at org.elasticsearch.plugins.PluginsService.loadBundles(PluginsService.java:419) ~[storm_hello_world-0.0.1-SNAPSHOT.jar:?]
... 10 more

关于如何解决此问题的任何想法？

提前感谢您的帮助！

Answer 1

Kryo 2.21 嵌入（通过 shade 插件）库 minlog。你可以安全地从你的发行版中删除这个 jar (/etc/storm/storm-0.10.0.nimbus/lib/minlog-1.2.jar)

Kryo 3.x 中移除了他的依赖项的嵌入