在 Confluent Connect 中创建连接器时出现 NoClassDefFoundError: org/apache/kafka/connect/header/ConnectHeaders

Question

我使用此页面上的说明在CentOS 7.9 上安装了融合平台。 sudo yum install confluent-platform-oss-2.11

我正在使用 AWS MSK 集群和 apache 版本 2.6.1。

我开始使用/usr/bin/connect-distributed /etc/kafka/connect-distributed.properties 进行连接。我在distributed.properties 中提供了 MSK 客户端端点作为引导程序。连接启动就好了。但是，当我尝试添加以下连接器时，它会引发以下错误。

连接器配置 -

    {
        "name": "source",
        "config": {
                "connector.class": "io.debezium.connector.postgresql.PostgresConnector",
                "tasks.max": "1",
                "plugin.name":"pgoutput",
                "database.hostname": "stage-sub.rds.amazonaws.com",
                "database.port": "5432",
                "database.user": "abc",
                "database.password": "abc",
                "database.server.name": "staging-orders",
                "database.dbname": "abc",
                "table.whitelist": "abc",
                "database.history.kafka.bootstrap.servers": "abcmigrat.3kppd9.c16.kafka.us-east-1.amazonaws.com:9092",
                "database.history.kafka.topic": "schema-changes.abc",
                "transforms": "route",
                "transforms.route.type": "org.apache.kafka.connect.transforms.RegexRouter",
                "transforms.route.regex": "([^.]+)\\.([^.]+)\\.([^.]+)",
                "transforms.route.replacement": "$3"
        }
}

错误 sn-p -

ERROR WorkerSourceTask{id=deb-payment-staging-0} Task threw an uncaught and unrecoverable exception (org.apache.kafka.connect.runtime.WorkerTask:172)
org.apache.kafka.connect.errors.ConnectException: An exception occurred in the change event producer. This connector will be stopped.
    at io.debezium.pipeline.ErrorHandler.setProducerThrowable(ErrorHandler.java:42)
    at io.debezium.pipeline.ChangeEventSourceCoordinator.lambda$start$0(ChangeEventSourceCoordinator.java:130)
    at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
    at java.util.concurrent.FutureTask.run(FutureTask.java:266)
    at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
    at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
    at java.lang.Thread.run(Thread.java:748)
Caused by: java.lang.BootstrapMethodError: java.lang.NoClassDefFoundError: org/apache/kafka/connect/header/ConnectHeaders
    at io.debezium.pipeline.EventDispatcher$BufferingSnapshotChangeRecordReceiver.changeRecord(EventDispatcher.java:431)
    at io.debezium.pipeline.EventDispatcher$1.changeRecord(EventDispatcher.java:176)
    at io.debezium.relational.RelationalChangeRecordEmitter.emitReadRecord(RelationalChangeRecordEmitter.java:88)
    at io.debezium.relational.RelationalChangeRecordEmitter.emitChangeRecords(RelationalChangeRecordEmitter.java:48)
    at io.debezium.pipeline.EventDispatcher.dispatchSnapshotEvent(EventDispatcher.java:166)
    at io.debezium.relational.RelationalSnapshotChangeEventSource.createDataEventsForTable(RelationalSnapshotChangeEventSource.java:375)
    at io.debezium.relational.RelationalSnapshotChangeEventSource.createDataEvents(RelationalSnapshotChangeEventSource.java:306)
        at io.debezium.relational.RelationalSnapshotChangeEventSource.doExecute(RelationalSnapshotChangeEventSource.java:136)
    at io.debezium.pipeline.source.AbstractSnapshotChangeEventSource.execute(AbstractSnapshotChangeEventSource.java:69)
    at io.debezium.pipeline.ChangeEventSourceCoordinator.lambda$start$0(ChangeEventSourceCoordinator.java:113)
    ... 5 more
Caused by: java.lang.NoClassDefFoundError: org/apache/kafka/connect/header/ConnectHeaders

Answer 1

我不熟悉这个特定的连接器，但一种可能的解释是连接器版本和 kafka connect worker 版本之间的兼容性问题。

您需要查看连接器的文档并验证它支持的连接版本。