从您的问题和您的代码中,我根据您检索到的日期数量(这发生在实际工作开始之前)推断出,您将针对您拥有日期的次数执行一个步骤。
我建议更改设计。创建一个 java 类,它将日期作为列表获取,并根据该列表动态创建步骤。像这样的:
@EnableBatchProcessing
public class JobConfig {
@Autowired
private JobBuilderFactory jobBuilderFactory;
@Autowired
private StepBuilderFactory stepBuilderFactory;
@Autowired
private JobDatesCreator jobDatesCreator;
@Bean
public Job executeMyJob() {
List<Step> steps = new ArrayList<Step>();
for (String date : jobDatesCreator.getDates()) {
steps.add(createStep(date));
}
return jobBuilderFactory.get("executeMyJob")
.start(createParallelFlow(steps))
.end()
.build();
}
private Step createStep(String date){
return stepBuilderFactory.get("readStgDbAndExportMasterListStep" + date)
.chunk(your_chunksize)
.reader(your_reader)
.processor(your_processor)
.writer(your_writer)
.build();
}
private Flow createParallelFlow(List<Step> steps) {
SimpleAsyncTaskExecutor taskExecutor = new SimpleAsyncTaskExecutor();
// max multithreading = -1, no multithreading = 1, smart size = steps.size()
taskExecutor.setConcurrencyLimit(1);
List<Flow> flows = steps.stream()
.map(step -> new FlowBuilder<Flow>("flow_" + step.getName()).start(step).build())
.collect(Collectors.toList());
return new FlowBuilder<SimpleFlow>("parallelStepsFlow")
.split(taskExecutor)
.add(flows.toArray(new Flow[flows.size()]))
.build();
}
}
编辑:添加了“jobParameter”输入(方法也略有不同)
在类路径的某处添加以下示例 .properties 文件:
sql.statement="select * from awesome"
并将以下注释添加到您的 JobDatesCreator 类
@PropertySource("classpath:example.properties")
您也可以提供特定的 sql 语句作为命令行参数。来自 spring 文档:
您可以使用特定的命令行开关启动(例如 java -jar
app.jar --name="Spring")。
有关更多信息,请参阅http://docs.spring.io/spring-boot/docs/current/reference/html/boot-features-external-config.html
获取日期的类(为什么要为此使用 tasklet?):
@PropertySource("classpath:example.properties")
public class JobDatesCreator {
@Value("${sql.statement}")
private String sqlStatement;
@Autowired
private CommonExportFromStagingDbJobConfig commonJobConfig;
private List<String> dates;
@PostConstruct
private void init(){
// Execute your logic here for getting the data you need.
JdbcTemplate jdbcTemplate = new JdbcTemplate(commonJobConfig.onlineStagingDb);
// acces to your sql statement provided in a property file or as a command line argument
System.out.println("This is the sql statement I provided in my external property: " + sqlStatement);
// for now..
dates = new ArrayList<>();
dates.add("date 1");
dates.add("date 2");
}
public List<String> getDates() {
return dates;
}
public void setDates(List<String> dates) {
this.dates = dates;
}
}
我还注意到您有很多可以很容易重构的重复代码。现在对于每个作家,你都有这样的东西:
@Bean
public FlatFileItemWriter<MasterList> division10MasterListFileWriter() {
FlatFileItemWriter<MasterList> writer = new FlatFileItemWriter<>();
writer.setResource(new FileSystemResource(new File(outDir, MerchHierarchyConstants.DIVISION_NO_10 )));
writer.setHeaderCallback(masterListFlatFileHeaderCallback());
writer.setLineAggregator(masterListFormatterLineAggregator());
return writer;
}
考虑改用这样的东西:
public FlatFileItemWriter<MasterList> divisionMasterListFileWriter(String divisionNumber) {
FlatFileItemWriter<MasterList> writer = new FlatFileItemWriter<>();
writer.setResource(new FileSystemResource(new File(outDir, divisionNumber )));
writer.setHeaderCallback(masterListFlatFileHeaderCallback());
writer.setLineAggregator(masterListFormatterLineAggregator());
return writer;
}
由于并非所有代码都可用于正确复制您的问题,因此此答案是解决您的问题的建议/指示。