Spring Cloud Data Flow：在 Kubernetes 集群中运行示例应用程序“partitioned-batch-job”时出现 Docker URI 错误

Question

我修改了数据流示例应用程序partitioned-batch-job 以通过在集群中运行的 SCDF 服务器将其部署在 kubernetes 集群中。我使用仪表板将这个应用程序作为一项任务启动。该应用程序启动正常，但我看到我提供给 DeployerPartitionHandler 的 docker 资源的以下 URI 错误。我已经在错误堆栈下方显示了修改后的代码 sn-p 以供参考。感谢有关我是否使用正确的 URI 语法的任何输入，如果是，为什么主步骤无法使用提供的 docker 图像参考启动工作人员。

java.lang.IllegalArgumentException: Unable to get URI for class path resource [docker:vrajkuma/partitioned-batch-job:2.3.1-SNAPSHOT]
at org.springframework.cloud.deployer.spi.kubernetes.DefaultContainerFactory.create(DefaultContainerFactory.java:80) ~[spring-cloud-deployer-kubernetes-2.6.2.jar:2.6.2]
at org.springframework.cloud.deployer.spi.kubernetes.AbstractKubernetesDeployer.createPodSpec(AbstractKubernetesDeployer.java:210) ~[spring-cloud-deployer-kubernetes-2.6.2.jar:2.6.2]
at org.springframework.cloud.deployer.spi.kubernetes.KubernetesTaskLauncher.launch(KubernetesTaskLauncher.java:237) ~[spring-cloud-deployer-kubernetes-2.6.2.jar:2.6.2]
at org.springframework.cloud.deployer.spi.kubernetes.KubernetesTaskLauncher.launch(KubernetesTaskLauncher.java:119) ~[spring-cloud-deployer-kubernetes-2.6.2.jar:2.6.2]
at org.springframework.cloud.task.batch.partition.DeployerPartitionHandler.launchWorker(DeployerPartitionHandler.java:394) [spring-cloud-task-batch-2.3.1-SNAPSHOT.jar:2.3.1-SNAPSHOT]
at org.springframework.cloud.task.batch.partition.DeployerPartitionHandler.launchWorkers(DeployerPartitionHandler.java:313) [spring-cloud-task-batch-2.3.1-SNAPSHOT.jar:2.3.1-SNAPSHOT]
at org.springframework.cloud.task.batch.partition.DeployerPartitionHandler.handle(DeployerPartitionHandler.java:302) [spring-cloud-task-batch-2.3.1-SNAPSHOT.jar:2.3.1-SNAPSHOT]
at org.springframework.batch.core.partition.support.PartitionStep.doExecute(PartitionStep.java:106) [spring-batch-core-4.3.3.jar:4.3.3]
at org.springframework.batch.core.step.AbstractStep.execute(AbstractStep.java:208) [spring-batch-core-4.3.3.jar:4.3.3]
at org.springframework.batch.core.job.SimpleStepHandler.handleStep(SimpleStepHandler.java:152) [spring-batch-core-4.3.3.jar:4.3.3]
at org.springframework.batch.core.job.AbstractJob.handleStep(AbstractJob.java:413) [spring-batch-core-4.3.3.jar:4.3.3]
at org.springframework.batch.core.job.SimpleJob.doExecute(SimpleJob.java:136) [spring-batch-core-4.3.3.jar:4.3.3]
at org.springframework.batch.core.job.AbstractJob.execute(AbstractJob.java:320) [spring-batch-core-4.3.3.jar:4.3.3]
at org.springframework.batch.core.launch.support.SimpleJobLauncher$1.run(SimpleJobLauncher.java:149) [spring-batch-core-4.3.3.jar:4.3.3]
at org.springframework.core.task.SyncTaskExecutor.execute(SyncTaskExecutor.java:50) [spring-core-5.3.9.jar:5.3.9]
at org.springframework.batch.core.launch.support.SimpleJobLauncher.run(SimpleJobLauncher.java:140) [spring-batch-core-4.3.3.jar:4.3.3]

这是文件JobConfiguration.java 的修改代码。我刚刚用 docker 图像参考替换了 maven 资源。还修改了 pom.xml 依赖以使用 kubernetes 部署器（而不是本地）。

@Configuration
public class JobConfiguration {

private static final int GRID_SIZE = 4;
// @checkstyle:off
@Autowired
public JobBuilderFactory jobBuilderFactory;
@Autowired
public StepBuilderFactory stepBuilderFactory;
@Autowired
public DataSource dataSource;
@Autowired
public JobRepository jobRepository;
// @checkstyle:on
@Autowired
private ConfigurableApplicationContext context;
@Autowired
private DelegatingResourceLoader resourceLoader;
@Autowired
private Environment environment;

@Bean
public PartitionHandler partitionHandler(TaskLauncher taskLauncher, JobExplorer jobExplorer,
                                         TaskRepository taskRepository, DockerResourceLoader dockerResourceLoader) throws Exception {
    Resource resource = this.resourceLoader
        .getResource("docker:vrajkuma/partitioned-batch-job:2.3.1-SNAPSHOT");

    DeployerPartitionHandler partitionHandler =
        new DeployerPartitionHandler(taskLauncher, jobExplorer, resource, "workerStep", taskRepository);

引用的 repository:tag 位于 docker hub 中，它与我在仪表板中注册应用程序时提供的 URI 相同。 k8s 设置是在虚拟机上运行的本地裸机集群。通过提供的 helm chart 部署 SCDF 服务器。我能够在集群中运行其他示例应用程序（billsetuptask 和 billrun）而没有任何问题。

谢谢。

Answer 1

看起来 DelegatingResourceLoader (this.resourceLoader) 应该根据 URI 字符串中的方案返回一个 DockerResourceLoader。不确定它是否正在这样做。现在我刚刚更改了代码以显式使用 DockerResourceLoader bean，它可以工作。

    @Bean
    public DockerResourceLoader getDockerResourceLoader() {
        return new DockerResourceLoader();
    }
    
    @Bean
    public PartitionHandler partitionHandler(TaskLauncher taskLauncher, JobExplorer jobExplorer,
                                             TaskRepository taskRepository, DockerResourceLoader dockerResourceLoader) throws Exception {

        Resource resource = dockerResourceLoader.getResource("docker:vrajkuma/partitioned-batch-job:2.3.1-SNAPSHOT");

/*
        Resource resource = this.resourceLoader
            .getResource("docker:vrajkuma/partitioned-batch-job:2.3.1-SNAPSHOT");
 */

        DeployerPartitionHandler partitionHandler =
            new DeployerPartitionHandler(taskLauncher, jobExplorer, resource, "workerStep", taskRepository);
...