已部署服务器上的 Hazelcast 会抛出 java.lang.ClassNotFoundException 而本地不会

Question

我想将我的 Spring Boot 应用程序升级到 Java 11 和 tomcat 9，该应用程序使用 Hazelcast 3.12.9 作为兑现机制。当我在本地部署时，一切看起来都很好，并且缓存成功。但是当应用程序在集群上运行时，我从所有可用的 3 个节点收到以下错误：

com.hazelcast.nio.serialization.HazelcastSerializationException: java.lang.ClassNotFoundException: com.some.service.some.server.domain.ClassA
    at com.hazelcast.internal.serialization.impl.JavaDefaultSerializers$JavaSerializer.read(JavaDefaultSerializers.java:88)
    at com.hazelcast.internal.serialization.impl.JavaDefaultSerializers$JavaSerializer.read(JavaDefaultSerializers.java:77)
    at com.hazelcast.internal.serialization.impl.StreamSerializerAdapter.read(StreamSerializerAdapter.java:48)
    at com.hazelcast.internal.serialization.impl.AbstractSerializationService.toObject(AbstractSerializationService.java:187)
    at com.hazelcast.map.impl.proxy.MapProxySupport.toObject(MapProxySupport.java:1237)
    at com.hazelcast.map.impl.proxy.MapProxyImpl.get(MapProxyImpl.java:120)
    at com.hazelcast.spring.cache.HazelcastCache.lookup(HazelcastCache.java:162)
    at com.hazelcast.spring.cache.HazelcastCache.get(HazelcastCache.java:67)
    at com.some.service.some.server.domain.ClassACache.get(GlassACache.java:28)
    at com.some.service.some.server.domain.ClassAFacade.getClassA(ClassAFacade.java:203)
    at com.some.service.some.server.domain.ClassAFacade.getGlassA(ClassAFacade.java:185)
    at com.some.service.some.server.domain.ClassALogic.lambda$getClassAInParallel$1(ClassALogic.java:196)
    at java.base/java.util.stream.ReferencePipeline$3$1.accept(ReferencePipeline.java:195)
    at java.base/java.util.ArrayList$ArrayListSpliterator.forEachRemaining(ArrayList.java:1655)
    at java.base/java.util.stream.AbstractPipeline.copyInto(AbstractPipeline.java:484)
    at java.base/java.util.stream.AbstractPipeline.wrapAndCopyInto(AbstractPipeline.java:474)
    at java.base/java.util.stream.ReduceOps$ReduceTask.doLeaf(ReduceOps.java:952)
    at java.base/java.util.stream.ReduceOps$ReduceTask.doLeaf(ReduceOps.java:926)
    at java.base/java.util.stream.AbstractTask.compute(AbstractTask.java:327)
    at java.base/java.util.concurrent.CountedCompleter.exec(CountedCompleter.java:746)
    at java.base/java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:290)
    at java.base/java.util.concurrent.ForkJoinPool$WorkQueue.topLevelExec(ForkJoinPool.java:1020)
    at java.base/java.util.concurrent.ForkJoinPool.scan(ForkJoinPool.java:1656)
    at java.base/java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1594)
    at java.base/java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:183)
Caused by: java.lang.ClassNotFoundException: com.some.service.some.server.domain.ClassA
    at java.base/jdk.internal.loader.BuiltinClassLoader.loadClass(BuiltinClassLoader.java:581)
    at java.base/jdk.internal.loader.ClassLoaders$AppClassLoader.loadClass(ClassLoaders.java:178)
    at java.base/java.lang.ClassLoader.loadClass(ClassLoader.java:522)
    at com.hazelcast.nio.ClassLoaderUtil.tryLoadClass(ClassLoaderUtil.java:288)

Hazelcast 定制器：

@Configuration
public class ClassAHazelcastConfig {
    private static final MaxSizePolicy HAZELCAST_DEFAULT_MAX_SIZE_POLICY = MaxSizePolicy.PER_NODE;
    private static final EvictionPolicy HAZELCAST_DEFAULT_EVICTION_POLICY = EvictionPolicy.LRU;

    @Bean
    HazelcastConfigurationCustomizer customizer(CachePropertiesHolder cacheProperties) {
        return config -> {
            config.addMapConfig(new MapConfig()
                    .setName(CLASS_A_CACHE)
                    .setMaxSizeConfig(new MaxSizeConfig(cacheProperties.getMaxsize(), HAZELCAST_DEFAULT_MAX_SIZE_POLICY))
                    .setEvictionPolicy(HAZELCAST_DEFAULT_EVICTION_POLICY)
                    .setTimeToLiveSeconds(cacheProperties.getTtl()));

            config.getSerializationConfig().addSerializerConfig(
                    new SerializerConfig()
                            .setImplementation(new OptionalStreamSerializer())
                            .setTypeClass(Optional.class)
            );
        };
    }
}

@Configuration
@EnableConfigurationProperties(CachePropertiesHolder.class)
public class CacheConfig implements CachingConfigurer, EnvironmentAware, ApplicationContextAware {

    public static final String CLASS_A_CACHE = "CACHE_A";

    private Environment environment;
    private ApplicationContext applicationContext;

    @Override
    @Bean(name="cacheManager")
    public CacheManager cacheManager() {
        boolean cachingEnabled = Boolean.parseBoolean(environment.getProperty("cache.enabled"));
        if (cachingEnabled) {
            HazelcastInstance instance = (HazelcastInstance) applicationContext.getBean("hazelcastInstance");
            return new HazelcastCacheManager(instance);
        }
        return new NoOpCacheManager();
    }

    @Override
    public CacheResolver cacheResolver() {
        return new SimpleCacheResolver(Objects.requireNonNull(cacheManager()));
    }

    @Bean
    @Override
    public KeyGenerator keyGenerator() {
        return new SimpleKeyGenerator();
    }

    @Bean
    @Override
    public CacheErrorHandler errorHandler() {
        return new SimpleCacheErrorHandler();
    }

    @Override
    public void setEnvironment(@NotNull Environment environment) {
        this.environment = environment;
    }

    @Override
    public void setApplicationContext(@NotNull ApplicationContext applicationContext) throws BeansException {
        this.applicationContext = applicationContext;
    }
}

Java 8 和 tomcat 8 一切正常。

更新：

经过几天的调查，我发现这些异常被抛出到使用的并行流中的唯一地方。

return forkJoinPool.submit(() ->
                    items.parallelStream()
                            .map(item -> {
                                try {
                                    return biFunction.apply(item);
                                } catch (Exception e) {
                                    LOG.error("Error", e);
                                    return Optional.<Item>empty();
                                }
                            })

奇怪的是，在 Java 8 和 tomcat 8 中我没有遇到这个问题。

Answer 1

最终，它与 Hazelcast 完全无关。主要是 Java 11 与 Java 8 的区别。

在抛出异常的部分中，使用了从 Java 11 开始的 ForkJoinPool，不能保证确切的创建时间，并且它似乎没有与 spring 应用程序相同的类加载器。 (Classes with default access result in NoClassDefFound error at runtime in spring boot project for java 11)

我做了一个错误的假设，因为异常来自 Hazelcast，而且我还发现还有其他相关问题。

Answer 2

Hazelcast 提供两种部署模式：嵌入式模式和客户端-服务器模式。请查看relevant documentation section 了解更多信息。

在前一种情况下，您有一个单个 JVM，并且您的所有类都在您的类路径上。

在后一种情况下，您至少有 2 个 JVM，每个客户端一个，每个成员一个。您似乎忘记设置成员的类路径以引用ClassA。

如何设置类路径取决于你如何启动你的 Hazelcast 成员。大多数情况下有效的方法是使用CLASSPATH 环境变量。