【问题标题】:Scala SBT Assembly cannot merge due to de-duplication error in StaticLoggerBinder.class由于 StaticLoggerBinder.class 中的重复数据删除错误,Scala SBT 程序集无法合并
【发布时间】:2015-05-28 12:48:40
【问题描述】:

我的问题是我不能再使用 sbt-assembly 插件,因为在几个从事这个项目的人之间出现了某种依赖合并问题。

运行 'sbt assembly' 时的问题:

[error] 合并过程中遇到 3 个错误 java.lang.RuntimeException:去重:发现不同的文件内容 在下面的: /Users/aris.vlasakakis/.ivy2/cache/ch.qos.logback/logback-classic/jars/logback-classic-1.1.2.jar:org/slf4j/impl/StaticLoggerBinder.class /Users/aris.vlasakakis/.ivy2/cache/org.slf4j/slf4j-log4j12/jars/slf4j-log4j12-1.6.1.jar:org/slf4j/impl/StaticLoggerBinder.class 去重:在以下发现不同的文件内容: /Users/aris.vlasakakis/.ivy2/cache/ch.qos.logback/logback-classic/jars/logback-classic-1.1.2.jar:org/slf4j/impl/StaticMDCBinder.class /Users/aris.vlasakakis/.ivy2/cache/org.slf4j/slf4j-log4j12/jars/slf4j-log4j12-1.6.1.jar:org/slf4j/impl/StaticMDCBinder.class 去重:在以下发现不同的文件内容: /Users/aris.vlasakakis/.ivy2/cache/ch.qos.logback/logback-classic/jars/logback-classic-1.1.2.jar:org/slf4j/impl/StaticMarkerBinder.class /Users/aris.vlasakakis/.ivy2/cache/org.slf4j/slf4j-log4j12/jars/slf4j-log4j12-1.6.1.jar:org/slf4j/impl/StaticMarkerBinder.class 在 sbtassembly.Assembly$.applyStrategies(Assembly.scala:140) 在 sbtassembly.Assembly$.x$1$lzycompute$1(Assembly.scala:25) 在 sbtassembly.Assembly$.x$1$1(Assembly.scala:23) 在 sbtassembly.Assembly$.stratMapping$lzycompute$1(Assembly.scala:23) 在 sbtassembly.Assembly$.stratMapping$1(Assembly.scala:23) 在 sbtassembly.Assembly$.inputs$lzycompute$1(Assembly.scala:67) 在 sbtassembly.Assembly$.inputs$1(Assembly.scala:57)

...等等

I am using SBT-assembly 0.13.0, and here is the build.sbt

name := "metamorphosis"

version := "0.10.0"

scalaVersion := "2.10.4"

lazy val common = RootProject(file("../"))

val main = Project(id = "bridge", base = file(".")).dependsOn(common)

//excludeDependencies += "org.apache.logging.log4j"

resolvers ++= Seq(
//  "Akka Snapshot Repository" at "http://repo.akka.io/snapshots/",
  "Typesafe Releases" at "http://repo.typesafe.com/typesafe/releases/",
  "Sonatype OSS Releases"  at "http://oss.sonatype.org/content/repositories/releases/"
)

libraryDependencies ++= Seq(
  "com.github.nscala-time" %% "nscala-time" % "1.8.0",
  "com.sclasen" %% "akka-kafka" % "0.0.10" % "compile",
  "com.typesafe.akka" %% "akka-actor" % "2.3.2",
  "org.codehaus.groovy" % "groovy" % "2.3.7",
  "com.typesafe.scala-logging" %% "scala-logging-slf4j" % "2.1.2",
  "ch.qos.logback" % "logback-classic" % "1.1.2",
  "ch.qos.logback.contrib" % "logback-json-classic" % "0.1.2",
  "ch.qos.logback.contrib" % "logback-jackson" % "0.1.2",
  "org.slf4j" % "slf4j-api" % "1.7.7",
  "com.fasterxml.jackson.core" % "jackson-databind" % "2.2.2",
  "org.clapper" %% "argot" % "1.0.3",
  "com.typesafe" % "config" % "1.2.1",
  "net.ceedubs" %% "ficus" % "1.0.1",
  "com.typesafe.play" %% "anorm" % "2.3.6",
  "org.json4s" %% "json4s-native" % "3.2.10",
  "org.json4s" %% "json4s-jackson" % "3.2.10",
  "com.github.tototoshi" %% "scala-csv" % "1.1.2",
  "org.scalatest" %% "scalatest" % "2.2.2",
  "junit" % "junit" % "4.11",
  "org.apache.kafka" %% "kafka" % "0.8.1.1"
    exclude("javax.jms", "jms")
    exclude("com.sun.jdmk", "jmxtools")
    exclude("com.sun.jmx", "jmxri")
    exclude("org.slf4j", "slf4j-simple")
)

mainClass in assembly := Some("com.company.kafka.agent.MetamorphosisActor")

关于如何解决这个合并问题的任何线索?

谢谢!

【问题讨论】:

    标签: scala sbt sbt-assembly


    【解决方案1】:

    问题在于,多个依赖项会引入slf4j 传递依赖项,从而导致冲突。最好的办法是排除这种传递依赖,而不是使用 SBT 程序集的合并策略来解决冲突。

    首先,您需要可视化 slf4j 的来源(对于所有顶级依赖项)。您可以使用像 sbt-dependency-graph 这样的工具来获取所有依赖项的可视化图表。或者,您可以从 SBT 命令提示符浏览 SBT 设置,或使用像 this onethis one 这样的 maven 站点。

    一旦找到重复的依赖项,请使用exclude,就像在构建文件中使用的一样。只留下一个 slf4j 传递依赖。您还可以排除所有 slf4j 传递依赖项(请参阅 here 方法)并在顶层手动添加 slf4j 依赖项。

    请注意,如果您的依赖项需要多个不同版本的 slf4j,则选择适用于所有情况的单个版本可能会出现问题。取决于二进制兼容性。

    【讨论】:

    • 为什么,这不是sbt run中同一个项目的问题,而只是在项目的打包中?
    • @matanster 当 SBT 打包时它会显式检查冲突,但当它只是运行时它只使用类路径。在 JVM 中,这种情况也不例外——路径上的第一类获胜。
    【解决方案2】:

    上一个答案提供了所有必需的反馈,但实际上并没有给出任何解决方案或尝试的步骤。

    我的错误:

    [error] (project/*:assembly) deduplicate: different file contents found in the following:
    [error] /home/user/.ivy2/cache/ch.qos.logback/logback-classic/jars/logback-classic-1.2.3.jar:org/slf4j/impl/StaticLoggerBinder.class
    [error] /home/user/.ivy2/cache/org.slf4j/slf4j-log4j12/jars/slf4j-log4j12-1.7.10.jar:org/slf4j/impl/StaticLoggerBinder.class
    [error] deduplicate: different file contents found in the following:
    [error] /home/user/.ivy2/cache/ch.qos.logback/logback-classic/jars/logback-classic-1.2.3.jar:org/slf4j/impl/StaticMDCBinder.class
    [error] /home/user/.ivy2/cache/org.slf4j/slf4j-log4j12/jars/slf4j-log4j12-1.7.10.jar:org/slf4j/impl/StaticMDCBinder.class
    [error] deduplicate: different file contents found in the following:
    [error] /home/user/.ivy2/cache/ch.qos.logback/logback-classic/jars/logback-classic-1.2.3.jar:org/slf4j/impl/StaticMarkerBinder.class
    [error] /home/user/.ivy2/cache/org.slf4j/slf4j-log4j12/jars/slf4j-log4j12-1.7.10.jar:org/slf4j/impl/StaticMarkerBinder.class
    

    推荐方式

    追踪冲突的依赖关系。例如,我不想要这些类:

    org.slf4j/slf4j-log4j12/jars/slf4j-log4j12-1.7.10.jar:org/slf4j/impl/StaticLoggerBinder.class
    org.slf4j/slf4j-log4j12/jars/slf4j-log4j12-1.7.10.jar:org/slf4j/impl/StaticMDCBinder.class
    org.slf4j/slf4j-log4j12/jars/slf4j-log4j12-1.7.10.jar:org/slf4j/impl/StaticMarkerBinder.class
    

    致电sbt(并确保使用sbt-dependency-graph插件):

    whatDependsOn org.slf4j slf4j-log4j12 1.7.10
    

    返回一个列表:

    [info] org.slf4j:slf4j-log4j12:1.7.10
    [info]   +-org.apache.hadoop:hadoop-auth:2.8.0
    [info]   | +-org.apache.hadoop:hadoop-common:2.8.0
    [info]   |   +-com.github.atais:test_2.11:0.0.3 [S]
    [info]   |   
    [info]   +-org.apache.hadoop:hadoop-common:2.8.0
    [info]   | +com.github.atais:test_2.11:0.0.3 [S]
    [info]   | 
    [info]   +-org.apache.zookeeper:zookeeper:3.4.6
    [info]     +-org.apache.curator:curator-client:2.7.1
    [info]     | +-org.apache.curator:curator-framework:2.7.1
    [info]     | | +-org.apache.curator:curator-recipes:2.7.1
    [info]     | | | +-org.apache.hadoop:hadoop-common:2.8.0
    [info]     | | |   +-com.github.atais:test_2.11:0.0.3 [S]
    [info]     | | |   
    [info]     | | +-org.apache.hadoop:hadoop-auth:2.8.0
    [info]     | |   +-org.apache.hadoop:hadoop-common:2.8.0
    [info]     | |     +-com.github.atais:test_2.11:0.0.3 [S]
    [info]     | |     
    [info]     | +-org.apache.hadoop:hadoop-common:2.8.0
    [info]     |   +-com.github.atais:test_2.11:0.0.3 [S]
    [info]     |   
    [info]     +-org.apache.curator:curator-framework:2.7.1
    [info]     | +-org.apache.curator:curator-recipes:2.7.1
    [info]     | | +-org.apache.hadoop:hadoop-common:2.8.0
    [info]     | |   +-com.github.atais:test_2.11:0.0.3 [S]
    [info]     | |   
    [info]     | +-org.apache.hadoop:hadoop-auth:2.8.0
    [info]     |   +-org.apache.hadoop:hadoop-common:2.8.0
    [info]     |     +-com.github.atais:test_2.11:0.0.3 [S]
    [info]     |     
    [info]     +-org.apache.curator:curator-recipes:2.7.1
    [info]     | +-org.apache.hadoop:hadoop-common:2.8.0
    [info]     |   +-com.github.atais:test_2.11:0.0.3 [S]
    [info]     |   
    [info]     +-org.apache.hadoop:hadoop-auth:2.8.0
    [info]     | +-org.apache.hadoop:hadoop-common:2.8.0
    [info]     |   +-com.github.atais:test_2.11:0.0.3 [S]
    [info]     |   
    [info]     +-org.apache.hadoop:hadoop-common:2.8.0
    [info]       +-com.github.atais:test_2.11:0.0.3 [S]
    

    我需要追踪hadoop-common 依赖关系,因为它是将com.github.atais:test_2.11:0.0.3 连接到不需要的org.slf4j:slf4j-log4j12:1.7.10 的依赖关系

    并修改它:

    libraryDependencies += "org.apache.hadoop" % "hadoop-common" % "2.8.1" exclude("org.slf4j", "slf4j-log4j12")
    

    简单的方法

    只需为冲突路径添加合并策略标志:

    assemblyMergeStrategy in assembly := {
      ...
      case PathList("org", "slf4j", xs@_*) => MergeStrategy.first
      case x => (assemblyMergeStrategy in assembly).value(x)
    }
    

    【讨论】:

    • 谢谢,你是我们配不上的英雄
    猜你喜欢
    • 2017-06-16
    • 2015-01-27
    • 2014-11-02
    • 2013-12-21
    • 2015-03-09
    • 2014-11-13
    • 2019-06-03
    • 2018-08-19
    • 2014-09-28
    相关资源
    最近更新 更多