【发布时间】:2015-06-28 08:09:14
【问题描述】:
我正在尝试使用 spark scala 将数据集保存在 Cassandra 数据库中,但是在运行代码时出现异常: 使用的链接:http://rustyrazorblade.com/2015/01/introduction-to-spark-cassandra/
error:
could not find implicit value for parameter rwf: com.datastax.spark.connector.writer.RowWriterFctory[FoodToUserIndex]
food_index.saveToCassandra("tutorial", "food_to_user_index")
^
.scala
def main(args: Array[String]): Unit = {
val conf = new SparkConf(true)
.set("spark.cassandra.connection.host", "localhost")
.set("spark.executor.memory", "1g")
.set("spark.cassandra.connection.native.port", "9042")
val sc = new SparkContext(conf)
case class FoodToUserIndex(food: String, user: String)
val user_table = sc.cassandraTable[CassandraRow]("tutorial", "user").select("favorite_food","name")
val food_index = user_table.map(r => new FoodToUserIndex(r.getString("favorite_food"), r.getString("name")))
food_index.saveToCassandra("tutorial", "food_to_user_index")}
build.sbt
name := "intro_to_spark"
version := "1.0"
scalaVersion := "2.11.2"
libraryDependencies += "org.apache.spark" %% "spark-core" % "1.2.0"
libraryDependencies += "com.datastax.spark" %% "spark-cassandra-connector" % "1.2.0-rc3"
如果将 scala 和 cassandra 连接器的版本更改为 2.10、1.1.0 就可以了。但我需要使用 scala 2.11:
scalaVersion := "2.10.4"
libraryDependencies += "org.apache.spark" %% "spark-core" % "1.2.0"
libraryDependencies += "com.datastax.spark" %% "spark-cassandra-connector" % "1.1.0" withSources() withJavadoc()
【问题讨论】:
-
哇............哇......哇......这个问题看起来像是“SO上最无法回答的问题”How to Ask
-
@nass:您分享的链接给出了 403。
-
@nass:请编辑您的问题并将您可能从链接中使用的代码添加到您的问题。此外,请更新您问题中的正确链接。
-
@LittlePanda OK 认为
标签: scala cassandra apache-spark