【问题标题】:Possible issue with Publish subscribe Channel发布订阅频道可能存在问题
【发布时间】:2019-07-11 08:55:28
【问题描述】:

我有一个应用程序可以监视特定 csv 文件 foo.csv 的 FTP 文件夹,一旦找到该文件,它就会将其拉到我的本地并生成新的输出格式 bar.csv,然后应用程序将发送新文件 @ 987654322@回到FTP文件夹并从本地删除。

现在我已经介绍了一个使用publishSubscribeChannel 的进程,它将文件转换为消息,然后使用jobLaunchingGateway,它将使用批处理读取finalBEY.csv 并将其打印到控制台,因为@987654326 它不起作用@ 在将其发送回 FTP 后从本地转发器中删除,我在第一个 jobLaunchingGateway 上使用 .channel("nullChannel") subscribe,它假设保留它直到收到批处理的回复,然后移动到下一个 subscribe 会将其发送到 ftp 并将其从本地删除,但似乎不是从本地删除它的情况,因此批处理找不到 finalBEY.csv 并引发我正在粘贴的错误下面是代码。

如果我从第二个 subscribe 中删除建议,它可以正常工作,因为这不会再从本地删除它。

你能帮忙解决这个问题吗?

public IntegrationFlow localToFtpFlow(Branch myBranch) {

        return IntegrationFlows.from(Files.inboundAdapter(new File(myBranch.getBranchCode()))
                        .filter(new ChainFileListFilter<File>()
                                .addFilter(new RegexPatternFileListFilter("final" + myBranch.getBranchCode() + ".csv"))
                                .addFilter(new FileSystemPersistentAcceptOnceFileListFilter(metadataStore(dataSource), "foo"))),//FileSystemPersistentAcceptOnceFileListFilter
                e -> e.poller(Pollers.fixedDelay(10_000)))
                .enrichHeaders(h ->h.headerExpression("file_originalFile", "new java.io.File('"+ myBranch.getBranchCode() +"/FEFOexport" + myBranch.getBranchCode() + ".csv')",true))
                .transform(p -> {
                    LOG.info("Sending file " + p + " to FTP branch " + myBranch.getBranchCode());
                    return p;
                })

                .log()
                .transform(m -> {
                            this.defaultSessionFactoryLocator.addSessionFactory(myBranch.getBranchCode(),createNewFtpSessionFactory(myBranch));
                            LOG.info("Adding factory to delegation");
                            return m;
                })
                .publishSubscribeChannel(s ->
                        s.subscribe(f ->f.transform(fileMessageToJobRequest())
                                        .handle(jobLaunchingGateway()).channel("nullChannel"))
                        .subscribe(h -> h.handle(Ftp.outboundAdapter(createNewFtpSessionFactory(myBranch), FileExistsMode.REPLACE)
                                         .useTemporaryFileName(true)
                                         .autoCreateDirectory(false)
                                         .remoteDirectory(myBranch.getFolderPath()), e -> e.advice(expressionAdvice()))))

                .get();
    }

    @Bean
    public FileMessageToJobRequest fileMessageToJobRequest(){
        FileMessageToJobRequest fileMessageToJobRequest = new FileMessageToJobRequest();
        fileMessageToJobRequest.setFileParameterName("file_path");
        fileMessageToJobRequest.setJob(orderJob);
        return fileMessageToJobRequest;
    }

    @Bean
    public JobLaunchingGateway jobLaunchingGateway() {
        SimpleJobLauncher simpleJobLauncher = new SimpleJobLauncher();
        simpleJobLauncher.setJobRepository(jobRepository);
        simpleJobLauncher.setTaskExecutor(new SyncTaskExecutor());
        JobLaunchingGateway jobLaunchingGateway = new JobLaunchingGateway(simpleJobLauncher);

        return jobLaunchingGateway;
    }

    /**
    * Creating the advice for routing the payload of the outbound message on different expressions (success, failure)
    * @return Advice
    */

    @Bean
    public Advice expressionAdvice() {
        ExpressionEvaluatingRequestHandlerAdvice advice = new ExpressionEvaluatingRequestHandlerAdvice();
        advice.setSuccessChannelName("success.input");
        advice.setOnSuccessExpressionString("payload.delete() + ' was successful'");
        //advice.setOnSuccessExpressionString("inputMessage.headers['file_originalFile'].renameTo(new java.io.File(payload.absolutePath + '.success.to.send'))");
        //advice.setFailureChannelName("failure.input");
        advice.setOnFailureExpressionString("payload + ' was bad, with reason: ' + #exception.cause.message");
        advice.setTrapException(true);
        return advice;
    }

这是错误,如您所见,第一行显示它已传输到 FTP,然后启动批处理作业,而在订阅中应先执行批处理...

 INFO 10452 --- [ask-scheduler-2] o.s.integration.ftp.session.FtpSession   : File has been successfully renamed from: /ftp/erbranch/EDMS/FEFO/finalBEY.csv.writing to /ftp/erbranch/EDMS/FEFO/finalBEY.csv

Caused by: java.lang.IllegalStateException: Input resource must exist (reader is in 'strict' mode): file [C:\Java Programs\spring4ftpappftp\BEY\finalBEY.csv]
    at org.springframework.batch.item.file.FlatFileItemReader.doOpen(FlatFileItemReader.java:251) ~[spring-batch-infrastructure-4.0.1.RELEASE.jar:4.0.1.RELEASE]
    at org.springframework.batch.item.support.AbstractItemCountingItemStreamItemReader.open(AbstractItemCountingItemStreamItemReader.java:146) ~[spring-batch-infrastructure-4.0.1.RELEASE.jar:4.0.1.RELEASE]
    ... 123 common frames omitted

调试代码:

2019-07-15 10:43:02.838  INFO 4280 --- [ask-scheduler-2] o.s.integration.ftp.session.FtpSession   : File has been successfully transferred to: /ftp/erbranch/EDMS/FEFO/finalBEY.csv.writing
2019-07-15 10:43:02.845  INFO 4280 --- [ask-scheduler-2] o.s.integration.ftp.session.FtpSession   : File has been successfully renamed from: /ftp/erbranch/EDMS/FEFO/finalBEY.csv.writing to /ftp/erbranch/EDMS/FEFO/finalBEY.csv
2019-07-15 10:43:02.845 DEBUG 4280 --- [ask-scheduler-2] o.s.b.f.s.DefaultListableBeanFactory     : Returning cached instance of singleton bean 'integrationEvaluationContext'
2019-07-15 10:43:02.848 DEBUG 4280 --- [ask-scheduler-2] o.s.b.f.s.DefaultListableBeanFactory     : Returning cached instance of singleton bean 'success.input'
2019-07-15 10:43:02.849 DEBUG 4280 --- [ask-scheduler-2] o.s.integration.channel.DirectChannel    : preSend on channel 'success.input', message: AdviceMessage [payload=true was successful, headers={id=eca55e1d-918e-3334-afce-66f8ab650748, timestamp=1563176582848}, inputMessage=GenericMessage [payload=BEY\finalBEY.csv, headers={file_originalFile=BEY\FEFOexportBEY.csv, id=a2f029b0-2609-1a11-67ef-4f56c7dd0752, file_name=finalBEY.csv, file_relativePath=finalBEY.csv, timestamp=1563176582787}]]
2019-07-15 10:43:02.849 DEBUG 4280 --- [ask-scheduler-2] o.s.i.t.MessageTransformingHandler       : success.org.springframework.integration.transformer.MessageTransformingHandler#0 received message: AdviceMessage [payload=true was successful, headers={id=eca55e1d-918e-3334-afce-66f8ab650748, timestamp=1563176582848}, 

inputMessage=GenericMessage [payload=BEY\finalBEY.csv, headers={file_originalFile=BEY\FEFOexportBEY.csv, id=a2f029b0-2609-1a11-67ef-4f56c7dd0752, file_name=finalBEY.csv, file_relativePath=finalBEY.csv, timestamp=1563176582787}]]

    2019-07-15 10:43:02.951 DEBUG 4280 --- [ask-scheduler-2] o.s.b.i.launch.JobLaunchingGateway       : jobLaunchingGateway received message: GenericMessage [payload=JobLaunchRequest: orderJob, parameters={file_path=C:\Java Programs\spring4ftpappftp\BEY\finalBEY.csv, dummy=1563176582946}, headers={file_originalFile=BEY\FEFOexportBEY.csv, id=c98ad6cb-cced-c911-1b93-9d054baeb9d0, file_name=finalBEY.csv, file_relativePath=finalBEY.csv, timestamp=1563176582951}]

2019-07-16 08:35:29.442  INFO 10208 --- [nio-8081-exec-3] o.s.i.channel.PublishSubscribeChannel    : Channel 'application.FTPOutp' has 1 subscriber(s).
2019-07-16 08:35:29.442  INFO 10208 --- [nio-8081-exec-3] o.s.i.endpoint.EventDrivenConsumer       : started 1o.org.springframework.integration.config.ConsumerEndpointFactoryBean#3
2019-07-16 08:35:29.442  INFO 10208 --- [nio-8081-exec-3] o.s.i.endpoint.EventDrivenConsumer       : Adding {message-handler} as a subscriber to the '1o.subFlow#1.channel#0' channel
2019-07-16 08:35:29.442  INFO 10208 --- [nio-8081-exec-3] o.s.integration.channel.DirectChannel    : Channel 'application.1o.subFlow#1.channel#0' has 1 subscriber(s).
2019-07-16 08:35:29.442  INFO 10208 --- [nio-8081-exec-3] o.s.i.endpoint.EventDrivenConsumer       : started 1o.subFlow#1.org.springframework.integration.config.ConsumerEndpointFactoryBean#1
2019-07-16 08:35:29.442  INFO 10208 --- [nio-8081-exec-3] o.s.i.endpoint.EventDrivenConsumer       : Adding {bridge} as a subscriber to the 'FTPOutp' channel
2019-07-16 08:35:29.442  INFO 10208 --- [nio-8081-exec-3] o.s.i.channel.PublishSubscribeChannel    : Channel 'application.FTPOutp' has 2 subscriber(s).

【问题讨论】:

    标签: spring-integration spring-batch


    【解决方案1】:

    由于您使用的是SyncTaskExecutor,批处理作业应该在调用线程上运行,然后是 FTP 适配器。

    使用 DEBUG 日志记录并按照消息流查看为什么没有发生。

    【讨论】:

    • 感谢 Gary,在调试之后,它看起来像是在执行 Ftp.outboundAdapter(createNewFtpSessionFactory(myBranch) 和 expressionAdvice(),它正在从本地文件夹中删除文件,因此不存在用于批处理,我不是能够知道它为什么这样做,但我假设当我使用前端表单进入 FTP 服务器并将其传递给此方法时,它首先启动 Ftp.outboundAdapter,我不知道逻辑但找不到任何其他原因,我为此添加了调试部分,也许您会看到不同的东西..thanks@Gary Russel
    • `jobLauncher.run(job, jobParameters)` 是一个 async 操作难道不是事实吗?您能否在消息到达publishSubscribeChannel() 时显示调试日志,以确定其订阅者被调用的顺序。您可以将id() 添加到该频道,以便更好地跟踪日志。
    • 应该是同步的,因为他使用的是SyncTaskExecutor。
    • 这很奇怪 - 你可以看到同一个 poller 线程同时运行。它们应该按顺序运行,因为订阅存储在LinkedHashMap 中。实际的订阅者是网桥——也许规范需要在网桥上显式设置order 属性?
    • 没问题,是的,在我检查了上面的修复后,这看起来很奇怪,我上传了一个不使用 Postgres 而是使用嵌入式 H2 的新副本,所以只需转到 localhost:8081,问题就会显示,如果我在expressionAdvice() 中评论advice.setOnSuccessExpressionString("payload.delete() + ' was successful'");,批处理将读取该文件,所以基本上它会在第一次订阅之前在第二次订阅中删除它...再次感谢您查看此@Gary Russell
    猜你喜欢
    • 1970-01-01
    • 2014-05-30
    • 2016-11-29
    • 1970-01-01
    • 1970-01-01
    • 1970-01-01
    • 1970-01-01
    • 2012-06-12
    • 1970-01-01
    相关资源
    最近更新 更多