【发布时间】:2022-02-16 10:51:27
【问题描述】:
我正在尝试将 debezium-connector-postgres 添加到我的 Kafka Connect。
首先我验证了我的配置
PUT http://localhost:8083/connector-plugins/io.debezium.connector.postgresql.PostgresConnector/config/validate
{
"name": "postgres-kafkaconnector",
"connector.class": "io.debezium.connector.postgresql.PostgresConnector",
"tasks.max": "1",
"database.hostname": "example.com",
"database.port": "5432",
"database.dbname": "my_db",
"database.user": "xxx",
"database.password": "xxx",
"database.server.name": "my_db_server",
"table.include.list": "public.products",
"plugin.name": "pgoutput"
}
它返回这个没有错误:
{
"name": "io.debezium.connector.postgresql.PostgresConnector",
"error_count": 0,
...
然后我尝试添加这个连接器
POST http://localhost:8083/connectors
{
"name": "postgres-kafkaconnector",
"config": {
"connector.class": "io.debezium.connector.postgresql.PostgresConnector",
"tasks.max": "1",
"database.hostname": "example.com",
"database.port": "5432",
"database.dbname": "my_db",
"database.user": "xxx",
"database.password": "xxx",
"database.server.name": "my_db_server",
"table.include.list": "public.products",
"plugin.name": "pgoutput"
}
}
连接器添加成功。
但是,我在运行时遇到了错误
GET http://localhost:8083/connectors/postgres-kafkaconnector/status
ERROR WorkerSourceTask{id=postgres-kafkaconnector-0} Task threw an uncaught and unrecoverable exception (org.apache.kafka.connect.runtime.WorkerTask:179)
io.debezium.jdbc.JdbcConnectionException: ERROR: syntax error
at io.debezium.connector.postgresql.connection.PostgresReplicationConnection.initPublication(PostgresReplicationConnection.java:180)
at io.debezium.connector.postgresql.connection.PostgresReplicationConnection.createReplicationSlot(PostgresReplicationConnection.java:351)
at io.debezium.connector.postgresql.PostgresConnectorTask.start(PostgresConnectorTask.java:136)
at io.debezium.connector.common.BaseSourceTask.start(BaseSourceTask.java:130)
at org.apache.kafka.connect.runtime.WorkerSourceTask.execute(WorkerSourceTask.java:208)
at org.apache.kafka.connect.runtime.WorkerTask.doRun(WorkerTask.java:177)
at org.apache.kafka.connect.runtime.WorkerTask.run(WorkerTask.java:227)
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
Caused by: org.postgresql.util.PSQLException: ERROR: syntax error
at org.postgresql.core.v3.QueryExecutorImpl.receiveErrorResponse(QueryExecutorImpl.java:2565)
at org.postgresql.core.v3.QueryExecutorImpl.processResults(QueryExecutorImpl.java:2297)
at org.postgresql.core.v3.QueryExecutorImpl.execute(QueryExecutorImpl.java:322)
at org.postgresql.jdbc.PgStatement.executeInternal(PgStatement.java:481)
at org.postgresql.jdbc.PgStatement.execute(PgStatement.java:401)
at org.postgresql.jdbc.PgStatement.executeWithFlags(PgStatement.java:322)
at org.postgresql.jdbc.PgStatement.executeCachedSql(PgStatement.java:308)
at org.postgresql.jdbc.PgStatement.executeWithFlags(PgStatement.java:284)
at org.postgresql.jdbc.PgStatement.executeQuery(PgStatement.java:236)
at io.debezium.connector.postgresql.connection.PostgresReplicationConnection.initPublication(PostgresReplicationConnection.java:137)
... 11 more
如何查看完整的日志,尤其是另外 11 行?我还检查了我的 Kubernetes pod 日志,它显示了相同的部分,但没有那 11 行。
当前的错误内容不是很有用。任何进一步调试的帮助将不胜感激!
更新 1:
GET http://localhost:8083/connectors
返回
[
"postgres-kafkaconnector"
]
【问题讨论】:
-
服务器日志可能会提供更多信息
-
是否要为 postgresql 实例添加第二个 Debezium 连接器?
-
@OneCricketeer 你的意思是 Postgres 日志吗?
-
@IskuskovAlexander 哦,这是我相信的第一个也是唯一一个 Debezium 连接器。
-
请提供
GET http://localhost:8083/connectors的输出
标签: postgresql apache-kafka apache-kafka-connect debezium change-data-capture