【发布时间】：2016-04-26 07:48:01
【问题描述】：
我正在尝试从 Spark 1.3 连接到 Cassandra 3.0。我知道 spark 中每个版本都有 Cassandra 连接器，但是 spark-cassandra-connector-java_2.10:1.3.0 连接器依赖于cassandra-driver-core:2.1.5，这就是为什么我使用最新的 cassandra 连接器，它依赖于最新的核心驱动程序。无论如何，到目前为止，这不是问题。我想问题是com.google.guava 包。
我的 pom 看起来像这样：
...
<dependency>
<groupId>com.datastax.spark</groupId>
<artifactId>spark-cassandra-connector-java_2.10</artifactId>
<version>1.5.0-M3</version>
</dependency>
<dependency>
<groupId>com.datastax.spark</groupId>
<artifactId>spark-cassandra-connector_2.10</artifactId>
<version>1.5.0-M3</version>
</dependency>
...
我已经从任何地方排除了谷歌番石榴：
<exclusions>
<exclusion>
<groupId>com.google.guava</groupId>
<artifactId>guava</artifactId>
</exclusion>
</exclusions>
所以在依赖树中只有 com.google.guava:guava:jar:16.0.1 在 com.datastax.spark:spark-cassandra-connector-java_2.10:jar:1.5.0-M3:compile 下。
但是我仍然收到以下错误：
yarn.ApplicationMaster: User class threw exception: Failed to open native connection to Cassandra at {139.19.52.111}:9042
java.io.IOException: Failed to open native connection to Cassandra at {139.19.52.111}:9042
at com.datastax.spark.connector.cql.CassandraConnector$.com$datastax$spark$connector$cql$CassandraConnector$$createSession(CassandraConnector.scala:162)
at com.datastax.spark.connector.cql.CassandraConnector$$anonfun$2.apply(CassandraConnector.scala:148)
at com.datastax.spark.connector.cql.CassandraConnector$$anonfun$2.apply(CassandraConnector.scala:148)
at com.datastax.spark.connector.cql.RefCountedCache.createNewValueAndKeys(RefCountedCache.scala:31)
at com.datastax.spark.connector.cql.RefCountedCache.acquire(RefCountedCache.scala:56)
at com.datastax.spark.connector.cql.CassandraConnector.openSession(CassandraConnector.scala:81)
at com.ambiverse.tagging.dao.impl.DAOCassandra.createTable(DAOCassandra.java:45)
at com.ambiverse.tagging.dao.impl.DAOCassandra.createTable(DAOCassandra.java:64)
at com.ambiverse.tagging.dao.impl.DAOCassandra.savePairRDD(DAOCassandra.java:70)
at com.ambiverse.tagging.statistics.entitycorrelation.CorrelationStatisticsSparkRunner.run(CorrelationStatisticsSparkRunner.java:176)
at com.ambiverse.tagging.statistics.entitycorrelation.CorrelationStatisticsSparkRunner.main(CorrelationStatisticsSparkRunner.java:94)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:497)
at org.apache.spark.deploy.yarn.ApplicationMaster$$anon$2.run(ApplicationMaster.scala:480)
Caused by: java.lang.NoSuchMethodError: com.google.common.util.concurrent.Futures.withFallback(Lcom/google/common/util/concurrent/ListenableFuture;Lcom/google/common/util/concurrent/FutureFallback;Ljava/util/concurrent/Executor;)Lcom/google/common/util/concurrent/ListenableFuture;
at com.datastax.driver.core.Connection.initAsync(Connection.java:178)
at com.datastax.driver.core.Connection$Factory.open(Connection.java:742)
at com.datastax.driver.core.ControlConnection.tryConnect(ControlConnection.java:240)
at com.datastax.driver.core.ControlConnection.reconnectInternal(ControlConnection.java:187)
at com.datastax.driver.core.ControlConnection.connect(ControlConnection.java:79)
at com.datastax.driver.core.Cluster$Manager.init(Cluster.java:1393)
at com.datastax.driver.core.Cluster.getMetadata(Cluster.java:402)
at com.datastax.spark.connector.cql.CassandraConnector$.com$datastax$spark$connector$cql$CassandraConnector$$createSession(CassandraConnector.scala:155)
在有人将我指向这篇博文寻求解决方案之前：http://arjon.es/2015/10/12/making-hadoop-2-dot-6-plus-spark-cassandra-driver-play-nice-together/，我使用 maven 作为构建工具，而不是 sbt。如果你知道我如何用 maven 做同样的事情，那就太好了。
【问题讨论】：
-
这很奇怪，对我来说，这个错误似乎表明你必须在 classpath 中有一个早于 14.0 的 guava 版本，因为那是在引入 withFallback 时。另一方面，我们在 datastax java-driver 中添加了一些验证，以在检测到旧版本的 guava 时抛出异常......
-
顺便说一句，如果我从 IDEA 启动 Spark，如何解决？
标签： maven hadoop apache-spark cassandra guava