Spring Boot：Spring Batch 作业不会自动启动答案

【问题标题】：Spring Boot : Spring Batch job does not launch automaticallySpring Boot：Spring Batch 作业不会自动启动
【发布时间】：2021-06-07 16:46:02
【问题描述】：

TL;DR

Spring Boot 批处理作业不会在 Windows 或 Spring Cloud Data Flow 中作为 uber jar 启动；然而，同样的代码在 Eclipse 中启动时也能正常工作。

我使用 Spring Boot 创建了一个简单的 Spring 批处理作业。我想在 Kubernetes 上运行这项工作。我按照this 指南中提到的有关如何在 SCDF（Spring Cloud 数据流）中启动 Spring 批处理作业的步骤进行操作。当我按照指南中所述注册此应用并创建任务时，会在 Kubernetes 中创建一个新 pod，并且应用程序运行良好；但是，批处理作业永远不会启动。这可能是什么原因？

我的 Spring Boot 批处理应用代码：

主类

@SpringBootApplication
public class BatchApplication {

    public static void main(String[] args) {
        SpringApplication.run(BatchApplication.class, args);
    }
}

作业配置

@Configuration
@EnableTask
@EnableBatchProcessing
@Profile("master")
public class BatchJob {

    @Autowired
    private JobBuilderFactory jobBuilderFactory;

    @Autowired
    private StepBuilderFactory stepBuilderFactory;

    public static final String JOB_NAME = "myJob";
    
    private static final Logger LOGGER = Logger.getLogger(BatchJob .class);

    @Bean(name = JOB_NAME )
    public Job job() {
        LOGGER.info("Creating job bean...");
        return jobBuilderFactory.get(JOB_NAME).incrementer(new RunIdIncrementer())
                .start(helloWorldStep()).build();
    }

    @Bean
    public Step helloWorldStep() {
        LOGGER.info("Creating Step bean...");
        return stepBuilderFactory.get("helloWorldStep").tasklet(helloWorldTasklet()).build();
    }

    @Bean
    public HelloWorldTasklet helloWorldTasklet() {
        LOGGER.info("Creating tasklet...");
        return new HelloWorldTasklet();
    }
}

HelloWorldTasklet

public class HelloWorldTasklet implements Tasklet {

    private static final Logger LOGGER = Logger.getLogger(HelloWorldTasklet.class);

    @Override
    public RepeatStatus execute(StepContribution contribution, ChunkContext chunkContext) throws Exception {

        LOGGER.info("Hello World");
        System.out.println("Hello World Again");
        return RepeatStatus.FINISHED;

    }

}

application.properties

spring.application.name=my-batch-app
spring.datasource.url=jdbc:h2:mem:dataflow
spring.datasource.driverClassName=org.h2.Driver
spring.datasource.username=sa
spring.datasource.password=
spring.jpa.database-platform=org.hibernate.dialect.H2Dialect
spring.cloud.task.closecontext_enabled=true

pom.xml 依赖项

<?xml version="1.0" encoding="UTF-8"?>
<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
    xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 https://maven.apache.org/xsd/maven-4.0.0.xsd">
    <modelVersion>4.0.0</modelVersion>
    <parent>
        <artifactId>my-project-parent</artifactId>
        <groupId>com.myapp</groupId>
        <version>MAR21_1.0</version>
    </parent>

    <groupId>com.myapp</groupId>
    <artifactId>my-batch-app</artifactId>

    <properties>
        <java.version>1.8</java.version>
    </properties>
    <dependencies>
        <dependency>
            <groupId>org.springframework.boot</groupId>
            <artifactId>spring-boot-starter-batch</artifactId>
            <version>2.3.11.RELEASE</version>
        </dependency>
        <dependency>
            <groupId>org.springframework.boot</groupId>
            <artifactId>spring-boot-starter-data-jpa</artifactId>
            <version>2.3.11.RELEASE</version>
        </dependency>

        <dependency>
            <groupId>org.springframework.boot</groupId>
            <artifactId>spring-boot-starter-test</artifactId>
            <version>2.3.11.RELEASE</version>
            <scope>test</scope>
            <exclusions>
                <exclusion>
                    <groupId>org.junit.vintage</groupId>
                    <artifactId>junit-vintage-engine</artifactId>
                </exclusion>
            </exclusions>
        </dependency>
         <dependency>
            <groupId>log4j</groupId>
            <artifactId>log4j</artifactId>
            <version>1.2.17-cloudera1</version>
        </dependency>
        <dependency>
            <groupId>com.h2database</groupId>
            <artifactId>h2</artifactId>
            <version>1.4.199</version>
            <!--<scope>runtime</scope>-->
        </dependency>
        
    </dependencies>

    <build>
        <plugins>
            <plugin>
                <groupId>org.springframework.boot</groupId>
                <artifactId>spring-boot-maven-plugin</artifactId>
                <version>2.1.0.RELEASE</version>
                <executions>
                    <execution>
                        <goals>
                            <goal>repackage</goal>
                        </goals>
                    </execution>
                </executions>
            </plugin>
        </plugins>
    </build>

</project>

my-project-parent 是我所有项目的父模块，其中仅包含 jar 版本的定义。但是，到目前为止，我并没有真正使用 my-batch-app 中这个 pom 中定义的任何这些版本。

<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
    xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
    <modelVersion>4.0.0</modelVersion>

    <groupId>com.myapp</groupId>
    <artifactId>my-project-parent</artifactId>
    <version>MAR21_1.0</version>
    <packaging>pom</packaging>
    <name>my-project-parent</name>

    <properties>
        <java.version>1.8</java.version>            
        <spring.kafka.version>2.5.5.RELEASE</spring.kafka.version>          <spring.cloud.starter.task.version>2.2.4.RELEASE</spring.cloud.starter.task.version>            <spring.boot.starter.data.jpa.version>2.2.1.RELEASE</spring.boot.starter.data.jpa.version>
        <database.h2.version>1.4.199</database.h2.version>          <spring.cloud.dataflow.rest.client.version>2.7.1</spring.cloud.dataflow.rest.client.version>
        <jackson-datatype-jsr310.version>2.10.0</jackson-datatype-jsr310.version>
        <spring.batch.version>4.2.4.RELEASE</spring.batch.version>
        <log4jversion>1.2.17-cloudera1</log4jversion>
    </properties>

    <build>
        <plugins>
            <!-- Maven Compiler Plugin -->
            <plugin>
                <groupId>org.apache.maven.plugins</groupId>
                <artifactId>maven-compiler-plugin</artifactId>
                <configuration>
                    <source>${java.version}</source>
                    <target>${java.version}</target>
                </configuration>
            </plugin>
        </plugins>
    </build>


</project>

我为上述应用程序注册了 docker 映像，创建了一个任务并从 SCDF UI 启动了该任务。我的 pod 日志中没有看到 Spring Batch 步骤或作业日志。事实上，我没有看到 System.out.println 或我在日志中的代码周围添加的所有日志语句。

pod 似乎正在运行 BatchApplication 类，但似乎根本没有启动作业。我错过了什么？ 即使我在 windows 上将 jar 作为 spring boot uber jar 运行，也会出现同样的问题

Picked up _JAVA_OPTIONS: -Djdk.tls.maxCertificateChainLength=20

  .   ____          _            __ _ _
 /\\ / ___'_ __ _ _(_)_ __  __ _ \ \ \ \
( ( )\___ | '_ | '_| | '_ \/ _` | \ \ \ \
 \\/  ___)| |_)| | | | | || (_| |  ) ) ) )
  '  |____| .__|_| |_|_| |_\__, | / / / /
 =========|_|==============|___/=/_/_/_/
 :: Spring Boot ::        (v2.2.5.RELEASE)

2021-06-07 16:23:58.021  INFO 1 --- [           main] .r.c.b.w.b.c.o.BatchApplication : Starting BatchApplication  on batch-controller-test-5wg6eryzpj with PID 1 (/tmp/my-batch-app.jar started by ? in /tmp)
2021-06-07 16:23:58.026  INFO 1 --- [           main] .r.c.b.w.b.c.o.BatchApplication  : The following profiles are active: master
2021-06-07 16:24:03.031  INFO 1 --- [           main] .s.d.r.c.RepositoryConfigurationDelegate : Bootstrapping Spring Data JPA repositories in DEFAULT mode.
2021-06-07 16:24:03.316  INFO 1 --- [           main] .s.d.r.c.RepositoryConfigurationDelegate : Finished Spring Data repository scanning in 185ms. Found 0 JPA repository interfaces.
2021-06-07 16:24:04.917  INFO 1 --- [           main] faultConfiguringBeanFactoryPostProcessor : No bean named 'errorChannel' has been explicitly defined. Therefore, a default PublishSubscribeChannel will be created.
2021-06-07 16:24:04.935  INFO 1 --- [           main] faultConfiguringBeanFactoryPostProcessor : No bean named 'taskScheduler' has been explicitly defined. Therefore, a default ThreadPoolTaskScheduler will be created.
2021-06-07 16:24:04.949  INFO 1 --- [           main] faultConfiguringBeanFactoryPostProcessor : No bean named 'integrationHeaderChannelRegistry' has been explicitly defined. Therefore, a default DefaultHeaderChannelRegistry will be created.
2021-06-07 16:24:06.814  INFO 1 --- [           main] trationDelegate$BeanPostProcessorChecker : Bean 'org.springframework.integration.config.IntegrationManagementConfiguration' of type [org.springframework.integration.config.IntegrationManagementConfiguration] is not eligible for getting processed by all BeanPostProcessors (for example: not eligible for auto-proxying)
2021-06-07 16:24:07.047  INFO 1 --- [           main] trationDelegate$BeanPostProcessorChecker : Bean 'integrationChannelResolver' of type [org.springframework.integration.support.channel.BeanFactoryChannelResolver] is not eligible for getting processed by all BeanPostProcessors (for example: not eligible for auto-proxying)
2021-06-07 16:24:07.113  INFO 1 --- [           main] trationDelegate$BeanPostProcessorChecker : Bean 'integrationDisposableAutoCreatedBeans' of type [org.springframework.integration.config.annotation.Disposables] is not eligible for getting processed by all BeanPostProcessors (for example: not eligible for auto-proxying)
2021-06-07 16:24:09.125  INFO 1 --- [           main] com.zaxxer.hikari.HikariDataSource       : HikariPool-1 - Starting...
2021-06-07 16:24:09.822  INFO 1 --- [           main] com.zaxxer.hikari.HikariDataSource       : HikariPool-1 - Start completed.
2021-06-07 16:24:10.512  INFO 1 --- [           main] o.hibernate.jpa.internal.util.LogHelper  : HHH000204: Processing PersistenceUnitInfo [name: default]
2021-06-07 16:24:11.220  INFO 1 --- [           main] org.hibernate.Version                    : HHH000412: Hibernate Core {5.4.8.Final}
2021-06-07 16:24:12.724  INFO 1 --- [           main] o.hibernate.annotations.common.Version   : HCANN000001: Hibernate Commons Annotations {5.1.0.Final}
2021-06-07 16:24:14.019  INFO 1 --- [           main] org.hibernate.dialect.Dialect            : HHH000400: Using dialect: org.hibernate.dialect.H2Dialect
2021-06-07 16:24:16.335  INFO 1 --- [           main] o.h.e.t.j.p.i.JtaPlatformInitiator       : HHH000490: Using JtaPlatform implementation: [org.hibernate.engine.transaction.jta.platform.internal.NoJtaPlatform]
2021-06-07 16:24:16.429  INFO 1 --- [           main] j.LocalContainerEntityManagerFactoryBean : Initialized JPA EntityManagerFactory for persistence unit 'default'
2021-06-07 16:24:19.814  INFO 1 --- [           main] o.s.s.c.ThreadPoolTaskScheduler          : Initializing ExecutorService 'taskScheduler'
2021-06-07 16:24:21.721  INFO 1 --- [           main] o.s.i.endpoint.EventDrivenConsumer       : Adding {logging-channel-adapter:_org.springframework.integration.errorLogger} as a subscriber to the 'errorChannel' channel
2021-06-07 16:24:21.721  INFO 1 --- [           main] o.s.i.channel.PublishSubscribeChannel    : Channel 'application.errorChannel' has 1 subscriber(s).
2021-06-07 16:24:21.722  INFO 1 --- [           main] o.s.i.endpoint.EventDrivenConsumer       : started bean '_org.springframework.integration.errorLogger'
2021-06-07 16:24:21.818  INFO 1 --- [           main] .r.c.b.w.b.c.o.BatchApplication  : Started BatchApplication  in 27.903 seconds (JVM running for 31.116)
2021-06-07 16:24:21.828  INFO 1 --- [extShutdownHook] o.s.i.endpoint.EventDrivenConsumer       : Removing {logging-channel-adapter:_org.springframework.integration.errorLogger} as a subscriber to the 'errorChannel' channel
2021-06-07 16:24:21.830  INFO 1 --- [extShutdownHook] o.s.i.channel.PublishSubscribeChannel    : Channel 'application.errorChannel' has 0 subscriber(s).
2021-06-07 16:24:21.830  INFO 1 --- [extShutdownHook] o.s.i.endpoint.EventDrivenConsumer       : stopped bean '_org.springframework.integration.errorLogger'
2021-06-07 16:24:21.832  INFO 1 --- [extShutdownHook] o.s.s.c.ThreadPoolTaskScheduler          : Shutting down ExecutorService 'taskScheduler'
2021-06-07 16:24:21.916  INFO 1 --- [extShutdownHook] j.LocalContainerEntityManagerFactoryBean : Closing JPA EntityManagerFactory for persistence unit 'default'
2021-06-07 16:24:21.917  INFO 1 --- [extShutdownHook] .SchemaDropperImpl$DelayedDropActionImpl : HHH000477: Starting delayed evictData of schema as part of SessionFactory shut-down'
2021-06-07 16:24:21.920  INFO 1 --- [extShutdownHook] com.zaxxer.hikari.HikariDataSource       : HikariPool-1 - Shutdown initiated...
2021-06-07 16:24:22.014  INFO 1 --- [extShutdownHook] com.zaxxer.hikari.HikariDataSource       : HikariPool-1 - Shutdown completed.

JAR 内容

为了让事情更清楚，这是我在 JAR 中看到的文件夹列表。看起来像使用spring boot maven插件来创建jar？那么为什么批处理作业不会在 Eclipse 之外启动：

--my-batch-app.jar
 - BOOT-INF
   - classes
   - lib
 - META-INF
 - org

注意：如您所见，日志还显示确实选择了主配置文件，因为我在启动任务时将--spring.profiles.active=master 作为输入参数传递了。

【问题讨论】：

我在您分享的内容中没有看到来自 Spring Batch 的任何日志。在创建 docker 映像并将您的应用程序部署到 Kubernetes 之前，您可以使用 java -jar myapp.jar 在本地运行您的应用程序并查看您的批处理作业是否启动？在继续使用 docker 和 kubernetes 之前，您需要确保在本地正确运行您的应用程序。
是的，我在发布问题之前已经这样做了。在本地以及独立作业（没有@EnableTask）上复制相同的行为。我开始认为这与我的主类不在顶级包中或者我需要显式添加 ComponentScan 的事实有关。现在试试。
抱歉我之前的评论中有错别字，我的意思是：“看看你的批处理作业是否启动了”。好的，请在本地测试并使用重现问题的最少代码更新您的问题。
上面的代码就完成了。我尝试将 jar 复制到另一个挂载点并运行它而不提供依赖 jar 的类路径。这证实了在类路径上没有找到 spring boot starter batch，因此没有自动启动 Job？我什至尝试显式提供启动批处理 jar 所在的依赖项文件夹的路径java -cp dependency/* -jar my-batch-app.jar，但这不起作用。
不，它不完整，pom 不显示插件定义。请检查：stackoverflow.com/help/minimal-reproducible-example。使用 spring boot maven 插件创建一个 uber-jar 并使用 java -jar myapp.jar 运行它：docs.spring.io/spring-boot/docs/current/maven-plugin/reference/…

标签： java spring-batch spring-cloud-dataflow spring-cloud-dataflow-ui

【解决方案1】：

几个后续问题-

您是按计划运行还是在启动时运行此应用？

您是否添加了此启动器依赖项？

<dependency>
  <groupId>org.springframework.boot</groupId>
  <artifactId>spring-boot-starter-batch</artifactId>
</dependency>

根据您的回答，如果您愿意分享，我可能需要深入研究您的代码。

【讨论】：

欢迎来到 Stack Overflow。请不要发布后续问题作为答案。现在回答您的后续问题，是的，我添加了依赖项。请参阅我对 pom 依赖项的问题的编辑。我只是从 UI 中单击“启动任务”选项，所以不确定为什么会安排它....

【解决方案2】：

我测试了您的应用程序，您的问题是您的 BatchJob 类未标记为 Spring 配置类。您应该按如下方式更新它：

// Your current imports
import org.springframework.context.annotation.Configuration;

@EnableTask
@EnableBatchProcessing
@Profile("master")
@Configuration // This is missing in your current config
public class BatchJob {
   ...
}

这样，我可以看到您的日志语句，并且作业在应用程序启动时正确启动。

【讨论】：

最终，Spring Boot 和 Java 代码都没有问题。原来我正在使用 maven 的复制依赖项指令。删除它解决了这个问题。我将进一步分析，看看最终 jar 在有和没有复制依赖项指令到 maven 的情况下有什么区别。在使用数据流在 K8 上运行相同的 jar 时，我还将检查这是否能解决问题..