(ActiveMQ-client-netty-threads)] org.apache.catalina.loader.Weba ..................: 此 Web 应用程序实例已停止

Question

我有一个连接到 ActiveMQ Artemis 的 spring-boot 应用程序。在 Tomcat 中部署时，它工作正常，但在部署同一应用程序的另一个版本并停止旧应用程序后，我在日志中不断收到此错误：

(ActiveMQ-client-netty-threads)] org.apache.catalina.loader.WebappClassLoaderBase.checkStateForResourceLoading
Illegal access: this web application instance has been stopped already. Could not load [io.netty.util.collection.IntObjectHashMap].
The following stack trace is thrown for debugging purposes as well as to attempt to terminate the thread which caused the illegal access. 
java.lang.IllegalStateException: Illegal access: this web application instance has been stopped already. Could not load [io.netty.util.collection.IntObjectHashMap]. The following stack trace is thrown for debugging purposes as well as to attempt to terminate the thread which caused the illegal access.
    at org.apache.catalina.loader.WebappClassLoaderBase.checkStateForResourceLoading(WebappClassLoaderBase.java:1311)
    at org.apache.catalina.loader.WebappClassLoaderBase.checkStateForClassLoading(WebappClassLoaderBase.java:1299)
    at org.apache.catalina.loader.WebappClassLoaderBase.loadClass(WebappClassLoaderBase.java:1158)
    at org.apache.catalina.loader.WebappClassLoaderBase.loadClass(WebappClassLoaderBase.java:1119)
    at org.apache.logging.log4j.util.LoaderUtil.loadClass(LoaderUtil.java:163)
    at org.apache.logging.log4j.core.impl.ThrowableProxy.loadClass(ThrowableProxy.java:582)
    at org.apache.logging.log4j.core.impl.ThrowableProxy.toExtendedStackTrace(ThrowableProxy.java:698)
    at org.apache.logging.log4j.core.impl.ThrowableProxy.<init>(ThrowableProxy.java:138)
    at org.apache.logging.log4j.core.impl.ThrowableProxy.<init>(ThrowableProxy.java:122)
    at org.apache.logging.log4j.core.impl.Log4jLogEvent.getThrownProxy(Log4jLogEvent.java:566)
    at org.apache.logging.log4j.core.pattern.ExtendedThrowablePatternConverter.format(ExtendedThrowablePatternConverter.java:64)
    at org.springframework.boot.logging.log4j2.ExtendedWhitespaceThrowablePatternConverter.format(ExtendedWhitespaceThrowablePatternConverter.java:53)
    at org.apache.logging.log4j.core.pattern.PatternFormatter.format(PatternFormatter.java:38)
    at org.apache.logging.log4j.core.layout.PatternLayout$PatternSerializer.toSerializable(PatternLayout.java:334)
    at org.apache.logging.log4j.core.layout.PatternLayout.toText(PatternLayout.java:233)
    at org.apache.logging.log4j.core.layout.PatternLayout.encode(PatternLayout.java:218)
    at org.apache.logging.log4j.core.layout.PatternLayout.encode(PatternLayout.java:58)
    at org.apache.logging.log4j.core.appender.AbstractOutputStreamAppender.directEncodeEvent(AbstractOutputStreamAppender.java:177)
    at org.apache.logging.log4j.core.appender.AbstractOutputStreamAppender.tryAppend(AbstractOutputStreamAppender.java:170)
    at org.apache.logging.log4j.core.appender.AbstractOutputStreamAppender.append(AbstractOutputStreamAppender.java:161)
    at org.apache.logging.log4j.core.config.AppenderControl.tryCallAppender(AppenderControl.java:156)
    at org.apache.logging.log4j.core.config.AppenderControl.callAppender0(AppenderControl.java:129)
    at org.apache.logging.log4j.core.config.AppenderControl.callAppenderPreventRecursion(AppenderControl.java:120)
    at org.apache.logging.log4j.core.config.AppenderControl.callAppender(AppenderControl.java:84)
    at org.apache.logging.log4j.core.config.LoggerConfig.callAppenders(LoggerConfig.java:448)
    at org.apache.logging.log4j.core.config.LoggerConfig.processLogEvent(LoggerConfig.java:433)
    at org.apache.logging.log4j.core.config.LoggerConfig.log(LoggerConfig.java:417)
    at org.apache.logging.log4j.core.config.LoggerConfig.log(LoggerConfig.java:403)
    at org.apache.logging.log4j.core.config.AwaitCompletionReliabilityStrategy.log(AwaitCompletionReliabilityStrategy.java:63)
    at org.apache.logging.log4j.core.Logger.logMessage(Logger.java:146)
    at org.apache.logging.log4j.spi.AbstractLogger.tryLogMessage(AbstractLogger.java:2163)
    at org.apache.logging.log4j.spi.AbstractLogger.logMessageTrackRecursion(AbstractLogger.java:2118)
    at org.apache.logging.log4j.spi.AbstractLogger.logMessageSafely(AbstractLogger.java:2101)
    at org.apache.logging.log4j.spi.AbstractLogger.logMessage(AbstractLogger.java:1995)
    at org.apache.logging.log4j.spi.AbstractLogger.logIfEnabled(AbstractLogger.java:1967)
    at org.apache.logging.slf4j.Log4jLogger.warn(Log4jLogger.java:259)
    at io.netty.util.internal.logging.Slf4JLogger.warn(Slf4JLogger.java:151)
    at io.netty.channel.epoll.EpollEventLoop.handleLoopException(EpollEventLoop.java:334)
    at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:328)
    at io.netty.util.concurrent.SingleThreadEventExecutor$5.run(SingleThreadEventExecutor.java:886)
    at java.lang.Thread.run(Thread.java:745)

2020-04-02 11:37:33.587  WARN 10941 --- [-netty-threads)] i.n.c.e.EpollEventLoop: Unexpected exception in the selector loop.
java.lang.NoClassDefFoundError: io/netty/util/collection/IntObjectHashMap$2
    at io.netty.util.collection.IntObjectHashMap.values(IntObjectHashMap.java:221) ~[netty-common-4.1.22.Final.jar:4.1.22.Final]
    at io.netty.channel.epoll.EpollEventLoop.closeAll(EpollEventLoop.java:355) [netty-transport-native-epoll-4.1.22.Final-linux-x86_64.jar:4.1.22.Final]
    at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:322) [netty-transport-native-epoll-4.1.22.Final-linux-x86_64.jar:4.1.22.Final]
    at io.netty.util.concurrent.SingleThreadEventExecutor$5.run(SingleThreadEventExecutor.java:886) [netty-common-4.1.22.Final.jar:4.1.22.Final]
    at java.lang.Thread.run(Thread.java:745) [?:1.8.0_111]

重启Tomcat后问题解决了，但我需要在不重启Tomcat的情况下解决这个问题，因为我不想重启生产Tomcat。

我在 spring-boot 应用中的 artemis 配置：

@Bean("connectionFactory")
public ConnectionFactory connectionFactory(JmsProperties appProperties) {
   ActiveMQConnectionFactory cf = new ActiveMQConnectionFactory(appProperties.getArtemis().getBrokerUrl());
   cf.setUser(appProperties.getArtemis().getUser());
   cf.setPassword(appProperties.getArtemis().getPassword());
   return cf;
}

我在这里创建 ActiveMQ 连接工厂，因为 spring-boot 只允许在 application.yml 中为 artemis 设置用户名和密码，但我需要在代理 url 中设置 consumerWindowSize，如下所示：tcp://localhost:61616?consumerWindowSize=0

Answer 1

如果 url 具有 tcp 模式，则连接工厂 ActiveMQConnection 使用 netty 连接到 ActiveMQ Artemis。当应用被tomcat停止时，在netty组上调用shutdownGracefully来停止所有的netty线程。

但是，shutdownGracefully 方法是异步的，所以 tomcat 在所有 netty 线程停止之前完成停止过程。如果 netty 线程在相关应用程序停止后尝试加载类，则 tomcat 类加载器会抛出 IllegalStateException 导致日志中的错误。

带有暂停的CachingConnectionFactory 应该可以缓解这个问题：

@Bean(destroyMethod = "destroy")
public ConnectionFactory connectionFactory(JmsProperties appProperties) {
    ActiveMQConnectionFactory cf = new ActiveMQConnectionFactory(appProperties.getArtemis().getBrokerUrl());
    cf.setUser(appProperties.getArtemis().getUser());
    cf.setPassword(appProperties.getArtemis().getPassword());

    return new CachingConnectionFactory(cf) {
        @Override
        public void destroy() {
            super.destroy();

            try {
                Thread.sleep(30000);
            } catch (InterruptedException e) {
                e.printStackTrace();
            }

            System.out.println("CachingConnectionFactory is destroyed!");
        }
    };
}