【发布时间】:2018-03-19 13:13:19
【问题描述】:
下面的代码在分配 H2OFrame 时抛出错误,很可能是隐式转换有问题。错误是:
类型不匹配;发现:org.apache.spark.h2o.RDD[Int](扩展 to) org.apache.spark.rdd.RDD[Int] 需要: org.apache.spark.h2o.H2OFrame(扩展为)water.fvec.H2OFrame
和代码:
import org.apache.spark.h2o._
import org.apache.spark._
import org.apache.spark.SparkContext._
object App1 extends App{
val conf = new SparkConf()
conf.setAppName("Test")
conf.setMaster("local[1]")
conf.set("spark.executor.memory","1g");
val sc = new SparkContext(conf)
val rawData = sc.textFile("c:\\spark\\data.csv")
val data = rawData.map(line => line.split(',').map(_.toDouble))
val response: RDD[Int] = data.map(row => row(0).toInt)
val h2oResponse: H2OFrame = response // <-- this line throws the error
sc.stop
}
【问题讨论】:
-
您是否尝试过导入
h2oContext.implicits._(它会将RDD隐式转换为H2OFrame。参考:docs.h2o.ai/sparkling-water/2.2/latest-stable/doc/tutorials/…
标签: scala apache-spark h2o