【问题标题】:zeppelin notebook "error: not found: value %"zeppelin 笔记本“错误:未找到:值 %”
【发布时间】:2016-08-14 08:29:03
【问题描述】:

根据reading csv in zeppelin 我应该使用%dep 加载csv jar,但我得到error: not found: value % 有人知道我缺少什么吗?

%spark

val a = 1

%dep
z.reset()
z.addRepo("Spark Packages Repo").url("http://dl.bintray.com/spark-packages/maven")
z.load("com.databricks:spark-csv_2.10:1.2.0")

a: Int = 1
<console>:28: error: not found: value %
              %dep
              ^

zeppelin 我看到的日志中:

 INFO [2016-04-21 11:44:19,300] ({pool-2-thread-11} SchedulerFactory.java[jobFinished]:137) - Job remoteInterpretJob_1461228259278 finished by scheduler org.apache.zeppelin.spark.SparkInterpreter1173192611
 INFO [2016-04-21 11:44:19,678] ({pool-2-thread-4} SchedulerFactory.java[jobStarted]:131) - Job remoteInterpretJob_1461228259678 started by scheduler org.apache.zeppelin.spark.SparkInterpreter1173192611
 INFO [2016-04-21 11:44:19,704] ({pool-2-thread-4} SchedulerFactory.java[jobFinished]:137) - Job remoteInterpretJob_1461228259678 finished by scheduler org.apache.zeppelin.spark.SparkInterpreter1173192611
 INFO [2016-04-21 11:44:36,968] ({pool-2-thread-12} SchedulerFactory.java[jobStarted]:131) - Job remoteInterpretJob_1461228276968 started by scheduler 1367682354
 INFO [2016-04-21 11:44:36,969] ({pool-2-thread-12} RReplInterpreter.scala[liftedTree1$1]:41) - intrpreting %dep
z.reset()
z.addRepo("Spark Packages Repo").url("http://dl.bintray.com/spark-packages/maven")
z.load("com.databricks:spark-csv_2.10:1.2.0")
ERROR [2016-04-21 11:44:36,975] ({pool-2-thread-12} RClient.scala[eval]:79) - R Error .zreplout <- rzeppelin:::.z.valuate(.zreplin) <text>:1:1: unexpected input
1: %dep
    ^
 INFO [2016-04-21 11:44:36,978] ({pool-2-thread-12} SchedulerFactory.java[jobFinished]:137) - Job remoteInterpretJob_1461228276968 finished by scheduler 1367682354
 INFO [2016-04-21 11:45:22,157] ({pool-2-thread-8} SchedulerFactory.java[jobStarted]:131) - Job remoteInterpretJob_1461228322157 started by scheduler org.apache.zeppelin.spark.SparkInterpreter1173192611

【问题讨论】:

  • 如果您不为每个解释器使用单独的单元格,通常会触发此错误。
  • 根据文档,%dep解释器应该在%spark%pyspark%sql之前使用。

标签: apache-spark apache-zeppelin


【解决方案1】:

每个单元格可以容纳一种类型的解释器。因此,为了使用%dep%spark,您应该在重新启动spark 解释器后将它们分成以%dep 开头的两个单元格,以便将其考虑在内。例如:

在第一个单元格中:

%dep
z.reset()
z.addRepo("Spark Packages Repo").url("http://dl.bintray.com/spark-packages/maven")
z.load("com.databricks:spark-csv_2.10:1.2.0")

现在您的依赖项已加载,您可以在不同的单元格中访问 spark 解释器:

%spark
val a = 1

PS:默认情况下,单元格使用 spark 解释器运行,因此您无需显式使用 %spark

【讨论】:

    猜你喜欢
    • 2017-01-29
    • 1970-01-01
    • 2016-12-25
    • 2018-06-27
    • 1970-01-01
    • 2018-05-08
    • 2016-06-23
    • 2020-10-12
    • 1970-01-01
    相关资源
    最近更新 更多