【问题标题】:intransitive dependencies with sbt assemblysbt 程序集的不及物依赖
【发布时间】:2016-06-20 07:03:18
【问题描述】:

我下载了一个(活动的)git 项目,它需要我安装 scala(也包括 sbt)和 hadoop。当我尝试通过 sbt(sbt 程序集)构建它时,它会产生大量关于 intransitive dependencies 的警告。我试图联系项目供应商,但他们没有给我适当的答复;只是告诉我它正在他们的机器上工作。

当我尝试“sbt assembly”时,我得到:

[info] Resolving jline#jline;2.12 ...
[info] Done updating.
[warn] Found intransitive dependency (org.apache.hadoop:hadoop-common:2.6.0) while publishMavenStyle is true, but Maven repositories
[warn]   do not support intransitive dependencies. Use exclusions instead so transitive dependencies
[warn]   will be correctly excluded in dependent projects.
[warn]            
[warn] Found intransitive dependency (org.apache.hadoop:hadoop-mapreduce-client-core:2.6.0) while publishMavenStyle is true, but Maven repositories
[warn]   do not support intransitive dependencies. Use exclusions instead so transitive dependencies
[warn]   will be correctly excluded in dependent projects.
[warn]            
[warn] Found intransitive dependency (org.apache.hadoop:hadoop-mapreduce-client-common:2.6.0) while publishMavenStyle is true, but Maven repositories
[warn]   do not support intransitive dependencies. Use exclusions instead so transitive dependencies
[warn]   will be correctly excluded in dependent projects.
[warn]            
[warn] Found intransitive dependency (org.apache.hadoop:hadoop-annotations:2.6.0) while publishMavenStyle is true, but Maven repositories
[warn]   do not support intransitive dependencies. Use exclusions instead so transitive dependencies
[warn]   will be correctly excluded in dependent projects.
[warn]            
[warn] Found intransitive dependency (org.apache.hadoop:hadoop-auth:2.6.0) while publishMavenStyle is true, but Maven repositories
[warn]   do not support intransitive dependencies. Use exclusions instead so transitive dependencies
[warn]   will be correctly excluded in dependent projects.
[warn]            
[warn] Found intransitive dependency (org.apache.hadoop:hadoop-yarn-common:2.6.0) while publishMavenStyle is true, but Maven repositories
[warn]   do not support intransitive dependencies. Use exclusions instead so transitive dependencies
[warn]   will be correctly excluded in dependent projects.
[warn]            
[warn] Found intransitive dependency (org.apache.hbase:hbase-server:0.96.2-hadoop2) while publishMavenStyle is true, but Maven repositories
[warn]   do not support intransitive dependencies. Use exclusions instead so transitive dependencies
[warn]   will be correctly excluded in dependent projects ...

还有一个 build.sbt 文件,其中包含项目所需的所有依赖项:

libraryDependencies ++= Seq(
            "com.beust" % "jcommander" % "1.35",
            "org.apache.hadoop" % "hadoop-common" % "2.6.0" intransitive(),
            "org.apache.hadoop" % "hadoop-mapreduce-client-core" % "2.6.0"  intransitive(),
            "org.apache.hadoop" % "hadoop-mapreduce-client-common" % "2.6.0"  intransitive(),
            "org.apache.hadoop" % "hadoop-annotations" % "2.6.0" intransitive(), 
            "org.apache.hadoop" % "hadoop-auth" % "2.6.0" intransitive(),
            "org.apache.hadoop" % "hadoop-yarn-common" % "2.6.0" intransitive(),
            "org.apache.hadoop" % "hadoop-yarn-api" % "2.6.0",
            "org.apache.hbase" % "hbase-server" % "0.96.2-hadoop2" intransitive(), 
            "org.apache.hbase" % "hbase-common" % "0.96.2-hadoop2"  intransitive(), 
            "org.apache.hbase" % "hbase-client" % "0.96.2-hadoop2" intransitive(),
            "org.apache.hbase" % "hbase-protocol" % "0.96.2-hadoop2",
            "org.cloudera.htrace" % "htrace-core" % "2.04" intransitive(),
            "commons-configuration" % "commons-configuration" % "1.6" exclude("commons-beanutils", "commons-beanutils-core"),
            "commons-httpclient" % "commons-httpclient" % "3.1",
            "commons-io" % "commons-io" % "2.4",
            "com.google.guava" % "guava" % "11.0.2",
            "log4j" % "log4j" % "1.2.16",
            "org.slf4j" % "slf4j-log4j12" % "1.7.5",
            "org.codehaus.jackson" % "jackson-jaxrs" % "1.9.13",
            "org.apache.avro" % "avro" % "1.7.4",
            "junit" % "junit" % "4.11" % "it,test",
            "com.novocode" % "junit-interface" % "0.10" % "it,test",
            "com.jsuereth" % "scala-arm_2.11" % "1.4" % "it,test",
            "org.scalatest" % "scalatest_2.11" % "2.2.1" % "it,test"
            )

assemblyMergeStrategy in assembly := {
    case PathList("org", "apache", "hadoop", "yarn", xs @ _*) => MergeStrategy.first 
    case x => val oldStrategy = (assemblyMergeStrategy in assembly).value
    oldStrategy(x)
}

assemblyExcludedJars in assembly := { 
  val cp = (fullClasspath in assembly).value
  cp filter {_.data.getName == "commons-beanutils-1.7.0.jar"}
}

scalaVersion := "2.11.4"

// output jar is here: target/ruleXtract.jar
assemblyOutputPath in assembly := file("target/ruleXtract.jar") 

// we want a jar without a main class so we can run it as "hadoop jar class args"
mainClass in (Compile, packageBin) := None

我应该怎么做才能让它工作?我不知道如何在没有“sbt assembly”的情况下构建这个项目。我查看了一些关于 sbt 的教程,并且 build.sbt 看起来不错——尽管不太确定。问题是什么?你有什么建议可以解决吗?

B.R.

【问题讨论】:

  • 为什么是 sbt 汇编?你试过 sbt 包吗?
  • 因为在教程的说明中它被写成'sbt assembly'。顺便说一句,仍然给了我相同的错误列表。

标签: scala hadoop sbt


【解决方案1】:

这可能无法解决您的问题,但会消除警告。当您想将项目发布到 Maven 存储库 (http://xerial.org/blog/2014/03/24/sbt/) 时,使用 SBT 属性“publishMavenStyle”。如果您不打算发布,可以将其设置为 false。

如果你正在使用多个 build.sbt 文件,你可以这样设置,这将为整个项目设置它:

publishMavenStyle in ThisBuild := false

【讨论】:

    猜你喜欢
    • 2015-04-28
    • 2010-09-06
    • 1970-01-01
    • 2022-08-11
    • 2015-10-04
    • 2013-08-02
    • 2016-07-27
    • 2017-04-08
    • 1970-01-01
    相关资源
    最近更新 更多