【问题标题】:embedded-kafka scalatest ClassNotFoundException: scala.collection.GenTraversableOnce嵌入式kafka scalatest ClassNotFoundException:scala.collection.GenTraversableOnce
【发布时间】:2020-10-01 20:35:10
【问题描述】:

Scala 新手,根据 README 顶部的第二个示例,我正在使用 https://github.com/embeddedkafka/embedded-kafka 实现 ScalaTest 的第一步:

import net.manub.embeddedkafka.EmbeddedKafka
import org.scalatest.matchers.should.Matchers
import org.scalatest.wordspec.AnyWordSpecLike

class MinimalTest extends AnyWordSpecLike with Matchers {

  "runs with embedded kafka" should {

    "work" in {
      EmbeddedKafka.start()

      1 + 1 shouldBe 2
      // ... code goes here

      EmbeddedKafka.stop()
    }
  }
}

运行这个测试,失败的级别比我熟悉的要低:

MinimalTest:
runs with embedded kafka
*** RUN ABORTED ***
  java.lang.NoClassDefFoundError: scala/collection/GenTraversableOnce
  at com.myorganization.api.MinimalTest.$anonfun$new$2(MinimalTest.scala:13)
  at scala.runtime.java8.JFunction0$mcV$sp.apply(JFunction0$mcV$sp.scala:18)
  at org.scalatest.OutcomeOf.outcomeOf(OutcomeOf.scala:85)
  at org.scalatest.OutcomeOf.outcomeOf$(OutcomeOf.scala:83)
  at org.scalatest.OutcomeOf$.outcomeOf(OutcomeOf.scala:104)
  at org.scalatest.Transformer.apply(Transformer.scala:22)
  at org.scalatest.Transformer.apply(Transformer.scala:20)
  at org.scalatest.wordspec.AnyWordSpecLike$$anon$3.apply(AnyWordSpecLike.scala:1076)
  at org.scalatest.TestSuite.withFixture(TestSuite.scala:196)
  at org.scalatest.TestSuite.withFixture$(TestSuite.scala:195)
  ...
  Cause: java.lang.ClassNotFoundException: scala.collection.GenTraversableOnce
  at java.base/jdk.internal.loader.BuiltinClassLoader.loadClass(BuiltinClassLoader.java:581)
  at java.base/jdk.internal.loader.ClassLoaders$AppClassLoader.loadClass(ClassLoaders.java:178)
  at java.base/java.lang.ClassLoader.loadClass(ClassLoader.java:522)
  at com.myorganization.api.MinimalTest.$anonfun$new$2(MinimalTest.scala:13)
  at scala.runtime.java8.JFunction0$mcV$sp.apply(JFunction0$mcV$sp.scala:18)
  at org.scalatest.OutcomeOf.outcomeOf(OutcomeOf.scala:85)
  at org.scalatest.OutcomeOf.outcomeOf$(OutcomeOf.scala:83)
  at org.scalatest.OutcomeOf$.outcomeOf(OutcomeOf.scala:104)
  at org.scalatest.Transformer.apply(Transformer.scala:22)
  at org.scalatest.Transformer.apply(Transformer.scala:20)
  ...

我怀疑依赖版本不匹配,但无法发现。这是我的相关 build.gradle 内容:

plugins {
    id 'java'
    id 'scala'
}

task spec(dependsOn: ['testClasses'], type: JavaExec) {
    main = 'org.scalatest.tools.Runner'
    args = ['-R', 'build/classes/scala/test', '-o']
    classpath = sourceSets.test.runtimeClasspath
}

dependencies {
    compile 'io.confluent:kafka-streams-avro-serde:5.4.0'
    compile 'io.github.embeddedkafka:embedded-kafka-streams_2.12:2.4.0'
    compile 'io.github.embeddedkafka:embedded-kafka_2.12:2.4.0'
    compile 'org.apache.avro:avro:1.9.1'
    compile 'org.apache.kafka:kafka-clients:2.4.0'
    compile 'org.apache.kafka:kafka-streams:2.4.0'
    compile 'org.apache.kafka:kafka_2.13:2.4.0'
    compile 'org.scala-lang:scala-reflect:2.12.6'
    testCompile 'io.github.embeddedkafka:embedded-kafka-schema-registry_2.12:5.4.0' // match schema registry version
    testCompile 'io.github.embeddedkafka:embedded-kafka-streams_2.13:2.4.0' // match kafka streams version
    testCompile 'io.github.embeddedkafka:embedded-kafka_2.13:2.4.0' // match kafka version
    testCompile 'org.scala-lang:scala-library:2.13.2'
    testCompile 'org.scalatest:scalatest_2.13:3.1.2'
    testImplementation 'junit:junit:4.11'
    testRuntime 'org.pegdown:pegdown:1.4.2'
}

【问题讨论】:

  • scala-reflect 2.13.2 而不是 2.12.6?另一件事是在您的测试用例中使用 scalatest 的 beforeandafterall
  • 谢谢!大问题...org.apache.kafka:kafka_2.13:2.4.0 中的 2.13 代表 Scala 2.13。所以所有这些都需要更新为2.12

标签: java scala gradle apache-kafka scalatest


【解决方案1】:

Gradle 对 Scala 的传递依赖确实有点有趣 - 从某种意义上说,它不会自动计算一组连贯的版本。

“缺失”类 scala/collection/GenTraversableOnce 是 scala-library 的一部分,在 2.12.x 中可用

所以你应该可以通过以下方式解决这个问题:

完成此操作后,假设您的 IDE 设置为与您的 build.gradle 文件同步,您应该能够查看 gradle 计算的依赖项(声明的和传递的)。如果您仍然有问题,请手动检查这些并查看 org.scala-lang:scala-library 库是否丢失或声明了两次。如果您有多个声明,您可以在 mvncentral.com 中查看每个库的依赖项。

注意步骤 (a) 的原因是您的“编译”依赖项中有 scala 人工制品,所以我假设您的所有代码(不仅仅是您的测试代码)都在使用 scala。

【讨论】:

    【解决方案2】:

    我可以确认修复是验证依赖项。或者更严格地检查测试依赖项。

    对我来说,我升级了 SpringBoot 版本,这改变了我的 spring-kafka-test 版本,其中实习生包括 kafka 2.13.x,最终包括 scala 库。

    我在项目的构建文件中使用了mvn dependency:tree,并搜索了“2.12”以查找旧依赖项的来源。示例依赖树(删除了不相关的依赖)。注意 org.apache.kafka:kafka_2.11:jar:0.10.0.0 包含在 my.company.riptide.api:ness-logger:jar:1.0.0 中,但 org.springframework.kafka:spring-kafka-test:jar:2.7.9 包含更新版本 org.apache.kafka:kafka_2.13:jar:2.7.2

    我的解决方案是从我的 ness-logger 依赖项中排除 kafaka_2.11,如下所示:

    <dependency>
      <groupId>my.company.riptide.api</groupId>
      <artifactId>ness-logger</artifactId>
      <version>1.0.0</version>
      <exclusions>
        <exclusion>
          <groupId>org.apache.kafka</groupId>
          <artifactId>kafka_2.11</artifactId>
        </exclusion>
      </exclusions>
    </dependency>
    

    mvn dependency:tree 的截断输出:

    [INFO] +- my.company.riptide.api:ness-logger:jar:1.0.0:compile
    [INFO] |  +- org.springframework.boot:spring-boot-starter-validation:jar:2.5.7:compile
    [INFO] |  |  +- org.apache.tomcat.embed:tomcat-embed-el:jar:9.0.55:compile
    [INFO] |  |  \- org.hibernate.validator:hibernate-validator:jar:6.2.0.Final:compile
    [INFO] |  |     \- jakarta.validation:jakarta.validation-api:jar:2.0.2:compile
    [INFO] |  +- commons-io:commons-io:jar:2.7:compile
    [INFO] |  +- my.company.eis:ness-logging-package:jar:4.0.1:compile
    [INFO] |  |  +- org.apache.avro:avro:jar:1.8.2:compile
    [INFO] |  |  |  +- org.codehaus.jackson:jackson-core-asl:jar:1.9.13:compile
    [INFO] |  |  |  +- org.codehaus.jackson:jackson-mapper-asl:jar:1.9.13:compile
    [INFO] |  |  |  +- com.thoughtworks.paranamer:paranamer:jar:2.7:compile
    [INFO] |  |  |  \- org.tukaani:xz:jar:1.5:compile
    [INFO] |  |  +- org.apache.avro:avro-compiler:jar:1.8.2:compile
    [INFO] |  |  |  +- org.apache.velocity:velocity:jar:1.7:compile
    [INFO] |  |  |  \- joda-time:joda-time:jar:2.7:compile
    [INFO] |  |  +- org.apache.kafka:kafka_2.11:jar:0.10.0.0:compile
    [INFO] |  |  |  +- com.101tec:zkclient:jar:0.8:compile
    [INFO] |  |  |  \- org.scala-lang.modules:scala-parser-combinators_2.11:jar:1.0.4:compile
    [INFO] |  |  \- com.netflix.hystrix:hystrix-core:jar:1.5.18:compile
    [INFO] |  |     +- com.netflix.archaius:archaius-core:jar:0.4.1:compile
    [INFO] |  |     \- io.reactivex:rxjava:jar:1.3.8:compile
    [INFO] |  +- org.springframework.boot:spring-boot-loader-tools:jar:2.5.7:compile
    [INFO] |  |  \- org.apache.commons:commons-compress:jar:1.21:compile
    [INFO] |  \- my.company.riptide.springboot:graceful-shutdown:jar:1.0.2:compile
    
    [INFO] +- org.springframework.kafka:spring-kafka:jar:2.7.9:compile
    [INFO] |  +- org.springframework:spring-messaging:jar:5.3.13:compile
    [INFO] |  +- org.springframework:spring-tx:jar:5.3.13:compile
    [INFO] |  +- org.springframework.retry:spring-retry:jar:1.3.1:compile
    [INFO] |  |  \- javax.annotation:javax.annotation-api:jar:1.3.2:compile
    [INFO] |  +- org.apache.kafka:kafka-clients:jar:2.7.2:compile
    [INFO] |  |  +- com.github.luben:zstd-jni:jar:1.4.5-6:compile
    [INFO] |  |  +- org.lz4:lz4-java:jar:1.7.1:compile
    [INFO] |  |  \- org.xerial.snappy:snappy-java:jar:1.1.7.7:compile
    [INFO] |  \- com.google.code.findbugs:jsr305:jar:3.0.2:compile
    [INFO] +- junit:junit:jar:4.13.2:test
    [INFO] |  \- org.hamcrest:hamcrest-core:jar:2.2:compile
    [INFO] +- io.cucumber:cucumber-spring:jar:7.0.0:test
    [INFO] |  \- org.apiguardian:apiguardian-api:jar:1.1.2:test
    [INFO] +- io.cucumber:cucumber-core:jar:7.0.0:test
    [INFO] |  +- io.cucumber:cucumber-gherkin:jar:7.0.0:test
    [INFO] |  +- io.cucumber:cucumber-gherkin-messages:jar:7.0.0:test
    [INFO] |  +- io.cucumber:messages:jar:17.1.1:test
    [INFO] |  +- io.cucumber:tag-expressions:jar:4.0.2:test
    [INFO] |  +- io.cucumber:cucumber-expressions:jar:13.0.1:test
    [INFO] |  +- io.cucumber:datatable:jar:7.0.0:test
    [INFO] |  +- io.cucumber:cucumber-plugin:jar:7.0.0:test
    [INFO] |  +- io.cucumber:docstring:jar:7.0.0:test
    [INFO] |  +- io.cucumber:html-formatter:jar:17.0.0:test
    [INFO] |  \- io.cucumber:create-meta:jar:6.0.1:test
    
    [INFO] +- org.springframework.kafka:spring-kafka-test:jar:2.7.9:test
    [INFO] |  +- org.apache.kafka:kafka-clients:jar:test:2.7.2:test
    [INFO] |  +- org.apache.kafka:kafka-streams:jar:2.7.2:test
    [INFO] |  |  +- org.apache.kafka:connect-json:jar:2.7.2:test
    [INFO] |  |  |  \- org.apache.kafka:connect-api:jar:2.7.2:test
    [INFO] |  |  \- org.rocksdb:rocksdbjni:jar:5.18.4:test
    [INFO] |  +- org.apache.kafka:kafka-streams-test-utils:jar:2.7.2:test
    [INFO] |  +- org.apache.kafka:kafka_2.13:jar:2.7.2:test
    [INFO] |  |  +- org.apache.kafka:kafka-raft:jar:2.7.2:test
    [INFO] |  |  +- com.fasterxml.jackson.module:jackson-module-scala_2.13:jar:2.12.5:test
    [INFO] |  |  +- com.fasterxml.jackson.dataformat:jackson-dataformat-csv:jar:2.12.5:test
    [INFO] |  |  +- net.sf.jopt-simple:jopt-simple:jar:5.0.4:compile
    [INFO] |  |  +- com.yammer.metrics:metrics-core:jar:2.2.0:compile
    [INFO] |  |  +- org.scala-lang.modules:scala-collection-compat_2.13:jar:2.2.0:test
    [INFO] |  |  +- org.scala-lang.modules:scala-java8-compat_2.13:jar:0.9.1:test
    [INFO] |  |  +- org.scala-lang:scala-library:jar:2.13.3:compile
    [INFO] |  |  +- org.scala-lang:scala-reflect:jar:2.13.3:test
    [INFO] |  |  +- com.typesafe.scala-logging:scala-logging_2.13:jar:3.9.2:test
    [INFO] |  |  +- org.apache.zookeeper:zookeeper:jar:3.5.9:compile
    [INFO] |  |  |  +- org.apache.zookeeper:zookeeper-jute:jar:3.5.9:compile
    [INFO] |  |  |  +- org.apache.yetus:audience-annotations:jar:0.5.0:compile
    [INFO] |  |  |  \- io.netty:netty-transport-native-epoll:jar:4.1.70.Final:compile
    [INFO] |  |  \- commons-cli:commons-cli:jar:1.4:test
    [INFO] |  +- org.apache.kafka:kafka_2.13:jar:test:2.7.2:test
    [INFO] |  \- org.junit.jupiter:junit-jupiter-api:jar:5.7.2:test
    [INFO] |     +- org.opentest4j:opentest4j:jar:1.2.0:test
    [INFO] |     \- org.junit.platform:junit-platform-commons:jar:1.7.2:test
    

    【讨论】:

      猜你喜欢
      • 1970-01-01
      • 1970-01-01
      • 1970-01-01
      • 2021-07-15
      • 2020-02-04
      • 1970-01-01
      • 2016-12-03
      • 2020-06-14
      • 1970-01-01
      相关资源
      最近更新 更多