dropfield 转换接收器连接器：(STRUCT) 类型没有到 SQL 数据库列类型的映射答案

【问题标题】：dropfield transform sink connector: (STRUCT) type doesn't have a mapping to the SQL database column typedropfield 转换接收器连接器：(STRUCT) 类型没有到 SQL 数据库列类型的映射
【发布时间】：2022-12-10 16:43:15
【问题描述】：

我创建了一个从 kafka 到 mysql 的接收器连接器。在接收器连接器的配置中进行转换并删除一些列后，我收到此错误，而无需转换它就可以工作：

(STRUCT) 类型没有到 SQL 数据库列类型的映射

{
    "name": "mysql-conf-sink",
    "config": {
      "connector.class": "io.confluent.connect.jdbc.JdbcSinkConnector",
      "tasks.max": "3",

      "value.converter": "io.confluent.connect.avro.AvroConverter",
      "value.converter.schema.registry.url": "http://localhost:8081",
      "topics": "mysql.cars.prices",
      "transforms": "dropPrefix,unwrap",

      "transforms.dropPrefix.type": "org.apache.kafka.connect.transforms.RegexRouter",
      "transforms.dropPrefix.regex": "mysql.cars.prices",
      "transforms.dropPrefix.replacement": "prices", 
  
      "transforms.timestamp.type": "org.apache.kafka.connect.transforms.TimestampConverter$Value",
      "transforms.timestamp.target.type": "Timestamp",
      "transforms.timestamp.field": "date_time",
      "transforms.timestamp.format": "yyyy-MM-dd HH:mm:ss",
  

      "errors.tolerance": "all",
      "errors.log.enable": "true",
      "errors.log.include.messages": "true",
  
  
      "connection.url": "jdbc:mysql://localhost:3306/product",
      "connection.user": "kafka",
      "connection.password": "123456",
  
      "transforms": "ReplaceField",
      "transforms.ReplaceField.type": "org.apache.kafka.connect.transforms.ReplaceField$Value",
      "transforms.ReplaceField.blacklist": "id, brand",


            
      "insert.mode": "insert",

      "auto.create": "true",
      "auto.evolve": "true",

      "batch.size": 50000


    }
  }

【问题讨论】：

标签： jdbc apache-kafka apache-kafka-connect

【解决方案1】：

您在 JSON 中多次输入 "transforms" 密钥，这是无效的。

尝试一个条目

"transforms": "unwrap,ReplaceField,dropPrefix",

您收到错误是因为您覆盖了该值，并且不再调用 unwrap，因此您嵌套了结构。

顺便说一句，黑名单属性已重命名为exclude - https://docs.confluent.io/platform/current/connect/transforms/replacefield.html#properties

【讨论】：