【发布时间】：2017-10-08 01:42:44
【问题描述】：
得到以下异常。我确保我在该位置有 jar 并且我以 root 权限运行该作业。
ERROR [ExecutorRunner for app-20170509111035-0004/19686] 2017-05-09 11:11:19,267 SPARK-WORKER Logging.scala:95 - Error running executor
java.lang.IllegalStateException: No assemblies found in '/usr/apps/cassandra/dse/resources/spark/lib'.
at org.apache.spark.launcher.CommandBuilderUtils.checkState(CommandBuilderUtils.java:249) ~[spark-launcher_2.10-1.6.3.3.jar:1.6.3.3]
at org.apache.spark.launcher.AbstractCommandBuilder.findAssembly(AbstractCommandBuilder.java:342) ~[spark-launcher_2.10-1.6.3.3.jar:1.6.3.3]
at org.apache.spark.launcher.AbstractCommandBuilder.buildClassPath(AbstractCommandBuilder.java:187) ~[spark-launcher_2.10-1.6.3.3.jar:1.6.3.3]
at org.apache.spark.launcher.AbstractCommandBuilder.buildJavaCommand(AbstractCommandBuilder.java:119) ~[spark-launcher_2.10-1.6.3.3.jar:1.6.3.3]
at org.apache.spark.launcher.WorkerCommandBuilder.buildCommand(WorkerCommandBuilder.scala:39) ~[spark-core_2.10-1.6.3.3.jar:1.6.3.3]
at org.apache.spark.launcher.WorkerCommandBuilder.buildCommand(WorkerCommandBuilder.scala:48) ~[spark-core_2.10-1.6.3.3.jar:1.6.3.3]
at org.apache.spark.deploy.worker.CommandUtils$.buildCommandSeq(CommandUtils.scala:63) ~[spark-core_2.10-1.6.3.3.jar:5.0.8]
at org.apache.spark.deploy.worker.CommandUtils$.buildProcessBuilder(CommandUtils.scala:51) ~[spark-core_2.10-1.6.3.3.jar:5.0.8]
at org.apache.spark.deploy.worker.ExecutorRunner.org$apache$spark$deploy$worker$ExecutorRunner$$fetchAndRunExecutor(ExecutorRunner.scala:143) ~[spark-core_2.10-1.6.3.3.jar:5.0.8]
at org.apache.spark.deploy.worker.ExecutorRunner$$anon$1.run(ExecutorRunner.scala:71) [spark-core_2.10-1.6.3.3.jar:5.0.8]
INFO [dispatcher-event-loop-0] 2017-05-09 11:11:19,268 SPARK-WORKER Logging.scala:58 - Executor app-20170509111035-0004/19686 finished with state FAILED message java.lang.IllegalStateException: No assemblies found in '/usr/apps/cassandra/dse/resources/spark/lib'.
任何帮助将不胜感激。
【问题讨论】：
-
是的。但它对我有帮助
-
感谢您的回复。我会检查并及时通知您。
-
您是如何运行 Spark 的 - 在 DSE 中，您必须使用
dse spark、dse spark-submit等运行 Spark。DSE 的依赖项布局与 OSS Spark 不同，因此如果您使用 OSS Spark 脚本运行你会喜欢有问题的。
标签： scala apache-spark cassandra datastax datastax-enterprise