【问题标题】:Spring Batch With Annotation and Caching带有注释和缓存的 Spring Batch
【发布时间】:2019-03-09 14:44:27
【问题描述】:

有没有人有很好的 Spring Batch 示例(使用注释)来缓存处理器可以访问的引用表?

我只需要一个简单的缓存,运行一个返回一些 byte[] 的查询并将其保存在内存中直到作业执行。

感谢有关此主题的任何帮助。

谢谢!

【问题讨论】:

    标签: spring-boot spring-batch spring-cache


    【解决方案1】:

    JobExecutionListener 可用于在作业执行前使用参考数据填充缓存,并在作业完成后清除缓存。

    这是一个例子:

    import org.springframework.batch.core.Job;
    import org.springframework.batch.core.JobExecution;
    import org.springframework.batch.core.JobExecutionListener;
    import org.springframework.batch.core.JobParameters;
    import org.springframework.batch.core.Step;
    import org.springframework.batch.core.StepContribution;
    import org.springframework.batch.core.configuration.annotation.EnableBatchProcessing;
    import org.springframework.batch.core.configuration.annotation.JobBuilderFactory;
    import org.springframework.batch.core.configuration.annotation.StepBuilderFactory;
    import org.springframework.batch.core.launch.JobLauncher;
    import org.springframework.batch.core.scope.context.ChunkContext;
    import org.springframework.batch.core.step.tasklet.Tasklet;
    import org.springframework.batch.repeat.RepeatStatus;
    import org.springframework.beans.factory.annotation.Autowired;
    import org.springframework.cache.CacheManager;
    import org.springframework.cache.concurrent.ConcurrentMapCacheManager;
    import org.springframework.context.ApplicationContext;
    import org.springframework.context.annotation.AnnotationConfigApplicationContext;
    import org.springframework.context.annotation.Bean;
    import org.springframework.context.annotation.Configuration;
    
    @Configuration
    @EnableBatchProcessing
    public class MyJob {
    
        private JobBuilderFactory jobs;
    
        private StepBuilderFactory steps;
    
        public MyJob(JobBuilderFactory jobs, StepBuilderFactory steps) {
           this.jobs = jobs;
           this.steps = steps;
        }
    
        @Bean
        public CacheManager cacheManager() {
            return new ConcurrentMapCacheManager(); // return the implementation you want
        }
    
        @Bean
        public Tasklet tasklet() {
            return new MyTasklet(cacheManager());
        }
    
        @Bean
        public Step step() {
            return steps.get("step")
                    .tasklet(tasklet())
                    .build();
        }
    
        @Bean
        public JobExecutionListener jobExecutionListener() {
            return new CachingJobExecutionListener(cacheManager());
        }
    
        @Bean
        public Job job() {
            return jobs.get("job")
                    .start(step())
                    .listener(jobExecutionListener())
                    .build();
        }
    
        class MyTasklet implements Tasklet {
    
            private CacheManager cacheManager;
    
            public MyTasklet(CacheManager cacheManager) {
                this.cacheManager = cacheManager;
            }
    
            @Override
            public RepeatStatus execute(StepContribution contribution, ChunkContext chunkContext) throws Exception {
                String name = (String) cacheManager.getCache("referenceData").get("foo").get();
                System.out.println("Hello " + name);
                return RepeatStatus.FINISHED;
            }
        }
    
        class CachingJobExecutionListener implements JobExecutionListener {
    
            private CacheManager cacheManager;
    
            public CachingJobExecutionListener(CacheManager cacheManager) {
                this.cacheManager = cacheManager;
            }
    
            @Override
            public void beforeJob(JobExecution jobExecution) {
                // populate cache as needed. Can use a jdbcTemplate to query the db here and populate the cache
                cacheManager.getCache("referenceData").put("foo", "bar");
            }
    
            @Override
            public void afterJob(JobExecution jobExecution) {
                // clear cache when the job is finished
                cacheManager.getCache("referenceData").clear();
            }
        }
    
        public static void main(String[] args) throws Exception {
            ApplicationContext context = new AnnotationConfigApplicationContext(MyJob.class);
            JobLauncher jobLauncher = context.getBean(JobLauncher.class);
            Job job = context.getBean(Job.class);
            jobLauncher.run(job, new JobParameters());
        }
    
    }
    

    执行时会打印:

    Hello bar
    

    这意味着数据已从缓存中正确检索。您需要调整示例以查询数据库并填充缓存(参见代码中的 cmets)。

    希望这会有所帮助。

    【讨论】:

    • 如果我用每个步骤所需的数据覆盖缓存中的数据,是否会影响作业重启能力。
    • 有什么方法可以应用多线程从 4 个表中读取数据,每个表包含 10 条 lacs 记录,然后我想用它嵌入另一个对象来创建最终对象?阅读每个表格需要 10 分钟,所以我想加快这个过程。对此有任何指导吗?
    【解决方案2】:

    您可以使用ehcache-jsr107 implementation. 快速设置。 Spring 和 ehcache 集成示例可用here。 您应该也可以使用 spring batch 设置相同的设置。

    希望能帮到你

    【讨论】:

      猜你喜欢
      • 2016-11-18
      • 2012-09-02
      • 1970-01-01
      • 2013-07-17
      • 1970-01-01
      • 2016-10-18
      • 2014-05-09
      • 2018-12-30
      • 2014-07-15
      相关资源
      最近更新 更多