【发布时间】:2016-11-16 22:01:16
【问题描述】:
我在db中有下表
+----------------+------------+------+-----+---------+----------------+
| Field | Type | Null | Key | Default | Extra |
+----------------+------------+------+-----+---------+----------------+
| id | bigint(20) | NO | PRI | NULL | auto_increment |
| VERSION | bigint(20) | NO | | NULL | |
| user_id | bigint(20) | NO | MUL | NULL | |
| measurement_id | bigint(20) | NO | MUL | NULL | |
| day | timestamp | NO | | NULL | |
| hour | tinyint(4) | NO | | NULL | |
| hour_timestamp | timestamp | NO | | NULL | |
| value | bigint(20) | NO | | NULL | |
+----------------+------------+------+-----+---------+----------------+
我正在尝试保存包含具有以下案例类结构的多行的 spark 数据框:
case class Record(val id : Int,
val VERSION : Int,
val user_id : Int,
val measurement_id : Int,
val day : Timestamp,
val hour : Int,
val hour_timestamp : Timestamp,
val value : Long )
当我尝试通过 jdbc 驱动程序将数据帧保存到我的 sql 时:
dataFrame.insertIntoJDBC(...)
我收到主键违规错误:
com.mysql.jdbc.exceptions.jdbc4.MySQLIntegrityConstraintViolationException: Duplicate entry '1' for key 'PRIMARY'
at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
at java.lang.reflect.Constructor.newInstance(Constructor.java:423)
我尝试将 id=0 设置为所有行的默认值,还尝试从案例类中删除 id 字段,但均未成功。
谁能帮忙?
谢谢, 托默
【问题讨论】:
-
你确定唯一的重复是第一条记录吗?
标签: jdbc apache-spark apache-spark-sql spark-dataframe