sbt.ResolveException 未解决的依赖

Question

我正在尝试处理 Twitter 流示例项目。我在定义 sbt 时遇到了问题。

我的 build.sbt

name := "Tutorial"
version := "0.1.0"
scalaVersion := "2.11.8"
retrieveManaged := true
libraryDependencies ++= Seq(
  "org.apache.spark" % "spark-core" % "2.11.0",
  "org.apache.spark" % "spark-streaming" % "1.1.0",
  "org.apache.spark" % "spark-streaming-twitter" % "1.1.0"
)

错误日志：

[warn]  Note: Some unresolved dependencies have extra attributes.  Check that these dependencies exist with the requested attributes.
[warn]      com.eed3si9n:sbt-assembly:0.9.2 (sbtVersion=0.11.3, scalaVersion=2.11.8)
[warn]      com.typesafe.sbteclipse:sbteclipse-plugin:2.2.0 (sbtVersion=0.11.3, scalaVersion=2.11.8)
[warn]      com.github.mpeltonen:sbt-idea:1.5.1 (sbtVersion=0.11.3, scalaVersion=2.11.8)
[warn] 
[error] {file:/home/muralee1857/scala/workspace/Tutorial/}default-109f4d/*:update: sbt.ResolveException: unresolved dependency: org.apache.spark#spark-core_2.11.8;1.5.1: not found
[error] unresolved dependency: com.eed3si9n#sbt-assembly;0.9.2: not found
[error] unresolved dependency: com.typesafe.sbteclipse#sbteclipse-plugin;2.2.0: not found
[error] unresolved dependency: com.github.mpeltonen#sbt-idea;1.5.1: not found

Answer 1

我建议您将依赖项的打包版本明确定义为

libraryDependencies ++= Seq(
  "org.apache.spark" % "spark-core_2.10" % "1.1.0",
  "org.apache.spark" % "spark-streaming_2.10" % "1.1.0" % "provided",
  "org.apache.spark" % "spark-streaming-twitter_2.10" % "1.1.0"
)

您可以在不定义packaged version 的情况下使用%%，但这会尝试在您的系统中下载package with scala version。有时sbt 找不到scala version packaged packages，这会产生依赖性问题。

Answer 2

这应该可行。请注意，我在这里使用%% 方法而不是%，以便在此处选择正确版本的 spark 库（scala 版本 2.11）。确保您将相同的 %% 函数应用于其他插件，如 sbt-assembly、sbt-assembly 等。

libraryDependencies ++= Seq(
  "org.apache.spark" %% "spark-core" % "2.11.0",
  "org.apache.spark" %% "spark-streaming" % "1.1.0",
  "org.apache.spark" %% "spark-streaming-twitter" % "1.1.0"
)