【发布时间】：2017-01-18 20:44:14
【问题描述】：
我在我的 ubuntu 14.04 上安装了 cassandra 2.1.11、spark 2.0.0.bin hadoop 2.7 和 java 版本 1.8.0_101。 对于 Spark Cassandra 连接器，我已经安装了 git
sudo apt-get install git
git clone https://github.com/datastax/spark-cassandra-connector.git
并构建它
cd spark-cassandra-connector
git checkout v1.4.0
./sbt/sbt assembly
并将scala的jar放在主目录
cp spark-cassandra-connector/target/scala-2.10/spark-cassandra-connector-assembly-1.4.0-SNAPSHOT.jar ~
并使用了连接器
bin/spark-shell --jars ~/spark-cassandra-connector-assembly-1.4.0-SNAPSHOT.jar
在 scala promt 中
sc.stop
import com.datastax.spark.connector._, org.apache.spark.SparkContext, org.apache.spark.SparkContext._, org.apache.spark.SparkConf
val conf = new SparkConf(true).set("spark.cassandra.connection.host", "localhost")
val sc = new SparkContext(conf)
我从 cqlsh 创建了 test 键空间和表 my_table 并测试连接，我运行了以下命令
eval test_spark_rdd = sc.cassandraTable("test", "my_table")
得到了错误
error: missing or invalid dependency detected while loading class file 'CassandraConnector.class'.
Could not access type Logging in package org.apache.spark,
because it (or its dependencies) are missing. Check your build definition for
missing or conflicting dependencies. (Re-run with `-Ylog-classpath` to see the problematic classpath.)
A full rebuild may help if 'CassandraConnector.class' was compiled against an incompatible version of org.apache.spark.
这是因为spark和cassandra的版本不匹配吗？
【问题讨论】：