【发布时间】:2015-05-03 18:59:59
【问题描述】:
我正在从 Oracle 中的一个表中读取数据并将整个转储插入 Db2。 表结构是相同的。我正在使用简单的 scala 类来完成上述任务。我已将插入批处理大小设置为 300。在更新了几批后,该类将抛出以下异常
com.ibm.db2.jcc.am.SqlIntegrityConstraintViolationException: Error for batch element #10: DB2 SQL Error: SQLCODE=-803, SQLSTATE=23505, SQLERRMC=1;PME.TM_ASSET_LQA_DETL, DRIVER=4.13.127
at com.ibm.db2.jcc.am.id.a(id.java:673) ~[db2jcc-4.13.127.jar:na]
at com.ibm.db2.jcc.am.id.a(id.java:60) ~[db2jcc-4.13.127.jar:na]
at com.ibm.db2.jcc.am.id.a(id.java:127) ~[db2jcc-4.13.127.jar:na]
at com.ibm.db2.jcc.t4.cb.a(cb.java:481) ~[db2jcc-4.13.127.jar:na]
at com.ibm.db2.jcc.t4.cb.a(cb.java:70) ~[db2jcc-4.13.127.jar:na]
at com.ibm.db2.jcc.t4.q.a(q.java:57) ~[db2jcc-4.13.127.jar:na]
at com.ibm.db2.jcc.t4.tb.a(tb.java:225) ~[db2jcc-4.13.127.jar:na]
at com.ibm.db2.jcc.am.oo.a(oo.java:3434) ~[db2jcc-4.13.127.jar:na]
at com.ibm.db2.jcc.am.oo.d(oo.java:5550) ~[db2jcc-4.13.127.jar:na]
at com.ibm.db2.jcc.am.oo.a(oo.java:4992) ~[db2jcc-4.13.127.jar:na]
at com.ibm.db2.jcc.am.oo.c(oo.java:4664) ~[db2jcc-4.13.127.jar:na]
at com.ibm.db2.jcc.am.oo.executeBatch(oo.java:2934) ~[db2jcc-4.13.127.jar:na]
at com.baml.regw.db.replicator.ReplicationRunnable$$anonfun$run$3.apply(SimpleReplicator.scala:105) ~[regw-db-replicator-0.0.933-SNAPSHOT.jar:na]
at com.baml.regw.db.replicator.ReplicationRunnable$$anonfun$run$3.apply(SimpleReplicator.scala:80) ~[regw-db-replicator-0.0.933-SNAPSHOT.jar:na]
由于异常与 IntegrityConstraint 有关,我尝试检查是否存在复合主键 (ID+TimeStamp+9999-12-31 00.00.000000),但该组合既不在 Oracle 表中,也不在 Db2 表中。 E Db2 表上的约束是
COLUMN NAME UNIQUE RULE
+ID+BUSINESS_STOP+BUSINESS_START Primary
+ID Duplicate
+BUSINESS_START Duplicate
+LOW_QUALITY_IND Duplicate
+IDENTIFIER1 Duplicate
+IDENTIFIER2 Duplicate
我浏览了 SO 中的其他此类问题,但没有一个修复对我有用。 执行此任务的代码
logger.info("Retrieving based on query string: " + queryStr + " for thread " + threadNum)
val start = System.currentTimeMillis()
val rs = stmt.executeQuery(queryStr)
val rsMd = rs.getMetaData()
val end = System.currentTimeMillis()
logger.info("Query execution time: " + (end - start) + "ms.")
done = true
var stmtCount = 0
Iterator.continually(rs).takeWhile(_.next()).foreach { rs =>
if (sourceConf.hasPath("blockSize")) {
done = false
}
//Subtract one to ignore the timestamp field that we are using
for (idx <- 1 to (rsMd.getColumnCount()-extraColumnCount)) {
try {
logger.info("destStmt.setObject"+rs.getObject(idx)+" column Type "+ rsMd.getColumnType(idx))
destStmt.setObject(idx, rs.getObject(idx), rsMd.getColumnType(idx))
}
catch {
case e:Exception => {
logger.warn("While attempting to set (1-based) index: " + idx +
" to value of type " + {if(rs.getObject(idx) != null) rs.getObject(idx).getClass().getName() else "[NULL]"} +
" received error: " + e.getMessage())
throw e
}
}
}
destStmt.addBatch()
stmtCount += 1
if(stmtCount % { if (destConf.hasPath("batchSize")) destConf.getInt("batchSize") else 200 } == 0) {
destStmt.executeBatch()
destDbConn.commit()
destStmt.clearBatch()
stmtCount = 0
}
}
if(stmtCount > 0) {
destStmt.executeBatch()
destDbConn.commit()
destStmt.clearBatch()
}
rs.close()
stmt.close()
【问题讨论】:
-
@Beryllium-感谢您的评论和指导,问题已解决。源表有重复的行,这就是异常背后的原因。再次感谢。