德比 Apache 随机 NullPointerException答案

【问题标题】：Derby Apache random NullPointerException德比 Apache 随机 NullPointerException
【发布时间】：2013-10-07 21:05:57
【问题描述】：

我们在使用嵌入式 Apache Derby 时遇到问题。无明显原因发生空指针异常。代码使用 EmbeddedSimpleDataSource 类来访问 DB 和 Statement 来执行查询。

应用程序在带有嵌入式 Derby 版本的 java Foundation profile/CDC 1.1（使用 cvm 而不是 jvm）上运行

奇怪的是这个问题完全是随机的（至少对我们来说，我们无法重现它）。从下面的日志中可以看出，异常发生在一个非常简单的选择查询上。

有时我们得到 3 次或多次异常，然后一切恢复正常。（很少对数据库的所有后续调用都失败，重新获得访问权限的唯一方法是重新启动应用程序（有时我们甚至必须删除数据库文件夹并重新创建它））

这是 derby 日志文件的摘录：

   Tue Oct 01 22:51:51 CEST 2013:
Booting Derby version The Apache Software Foundation - Apache Derby - 10.10.1.1 - (1458268): instance c013800d-0141-75ca-8140-000000068414 
on database directory /mnt/nand/lu/smarthubdb with class loader sun.misc.Launcher$AppClassLoader[ucp=sun.misc.URLClassPath[path=file:/mnt/nand/lu/SmartHub.jar],parent=sun.misc.Launcher$ExtClassLoader[ucp=sun.misc.URLClassPath[path=file:/usr/lib/ext/API_3.05.jar,file:/usr/lib/ext/sunjce_provider.jar],parent=null]] 
Loaded from file:/mnt/nand/lu/lib/derby.jar
java.vendor=Sun Microsystems Inc.
user.dir=/mnt/nand/lu
os.name=Linux
os.arch=sh3
os.version=2.6.25.9-svn298-dirty2
derby.system.home=null
Database Class Loader started - derby.database.classpath=''
Wed Oct 02 05:53:23 CEST 2013 Thread[main,5,main] (XID = 33536), (SESSIONID = 3), (DATABASE = smarthubdb), (DRDAID = null), Cleanup action starting
Wed Oct 02 05:53:23 CEST 2013 Thread[main,5,main] (XID = 33536), (SESSIONID = 3), (DATABASE = smarthubdb), (DRDAID = null), Failed Statement is: SELECT device.serial_number, tariff,reference_power_active, overload_power_limit_active, breaker_state,logical_device_type.name FROM device, logical_device_type WHERE device.id_logical_device_type = logical_device_type.id AND serial_number='041068350153'
java.lang.NullPointerException
    at org.apache.derby.exe.ac601a400fx0141x75cax8140x00000006841480.createResultSet(Compiled Method)(Unknown Source)
    at org.apache.derby.impl.sql.execute.CursorActivation.decorateResultSet(Unknown Source)
    at org.apache.derby.impl.sql.execute.BaseActivation.execute(Unknown Source)
    at org.apache.derby.impl.sql.GenericActivationHolder.execute(Unknown Source)
    at org.apache.derby.impl.sql.GenericPreparedStatement.executeStmt(Compiled Method)(Unknown Source)
    at org.apache.derby.impl.sql.GenericPreparedStatement.execute(Compiled Method)(Unknown Source)
    at org.apache.derby.impl.jdbc.EmbedStatement.executeStatement(Unknown Source)
    at org.apache.derby.impl.jdbc.EmbedStatement.execute(Compiled Method)(Unknown Source)
    at org.apache.derby.impl.jdbc.EmbedStatement.executeQuery(Unknown Source)
    at com.s.g.a.c(Unknown Source)
    at com.s.g.a.a(Unknown Source)
    at com.s.g.b.a(Unknown Source)
    at com.s.statemachine.b.e.b(Unknown Source)
    at com.s.statemachine.b.i.c(Compiled Method)(Unknown Source)
    at com.s.statemachine.SmartHub.A(Compiled Method)(Unknown Source)
    at com.s.statemachine.SmartHub.a(Unknown Source)
    at com.s.statemachine.SmartHub.main(Unknown Source)
    at sun.misc.CVM.runMain(Unknown Source)
Cleanup action completed
Wed Oct 02 06:15:26 CEST 2013 Thread[main,5,main] (XID = 34359), (SESSIONID = 7), (DATABASE = smarthubdb), (DRDAID = null), Cleanup action starting
Wed Oct 02 06:15:26 CEST 2013 Thread[main,5,main] (XID = 34359), (SESSIONID = 7), (DATABASE = smarthubdb), (DRDAID = null), Failed Statement is: SELECT device.serial_number, tariff,reference_power_active, overload_power_limit_active, breaker_state,logical_device_type.name FROM device, logical_device_type WHERE device.id_logical_device_type = logical_device_type.id AND serial_number='041067350131'
java.lang.NullPointerException
    at org.apache.derby.exe.ac601a400fx0141x75cax8140x00000006841485.createResultSet(Compiled Method)(Unknown Source)
    at org.apache.derby.impl.sql.execute.CursorActivation.decorateResultSet(Unknown Source)
    at org.apache.derby.impl.sql.execute.BaseActivation.execute(Unknown Source)
    at org.apache.derby.impl.sql.GenericActivationHolder.execute(Unknown Source)
    at org.apache.derby.impl.sql.GenericPreparedStatement.executeStmt(Compiled Method)(Unknown Source)
    at org.apache.derby.impl.sql.GenericPreparedStatement.execute(Unknown Source)
    at org.apache.derby.impl.jdbc.EmbedStatement.executeStatement(Unknown Source)
    at org.apache.derby.impl.jdbc.EmbedStatement.execute(Compiled Method)(Unknown Source)
    at org.apache.derby.impl.jdbc.EmbedStatement.executeQuery(Unknown Source)
    at com.s.g.a.c(Unknown Source)
    at com.s.g.a.a(Unknown Source)
    at com.s.g.b.a(Unknown Source)
    at com.s.statemachine.b.e.b(Unknown Source)
    at com.s.statemachine.b.i.c(Compiled Method)(Unknown Source)
    at com.s.statemachine.SmartHub.A(Compiled Method)(Unknown Source)
    at com.s.statemachine.SmartHub.a(Unknown Source)
    at com.s.statemachine.SmartHub.main(Unknown Source)
    at sun.misc.CVM.runMain(Unknown Source)
Cleanup action completed

这是表的结构：

private final static String               MCS_TABLE      = "metering_campaign_strategy";
private final static String[]             MCS_COLS       = { "serial_number",
            "measure_kind", "frequencyUnit", "frequency", "begin_date", "end_date", "last_retrieve_date" };
private final static String[]             MCS_COLS_TYPES = { "varchar(40) NOT NULL",
            "varchar(15) NOT NULL", "varchar(10) NOT NULL", "integer", "timestamp NOT NULL", "timestamp", "timestamp" }

;

如您所见，出现问题的表是以编程方式创建的。

我添加了 DumpClassFile 选项，我一得到结果就会更新这篇文章

根据要求，这是数据库的架构；

Wed Oct 02 18:53:14 CEST 2013 Thread[main,5,main] (XID = 165), (SESSIONID = 0), (DATABASE = smarthubdb), (DRDAID = null), Committing
Wed Oct 02 18:53:14 CEST 2013 Thread[main,5,main] (XID = 165), (SESSIONID = 0), (DATABASE = smarthubdb), (DRDAID = null), Rolling back
Wed Oct 02 18:53:14 CEST 2013 Thread[main,5,main] (XID = 166), (SESSIONID = 1), (DATABASE = smarthubdb), (DRDAID = null), Committing
Wed Oct 02 18:53:15 CEST 2013 Thread[main,5,main] (XID = 167), (SESSIONID = 1), (DATABASE = smarthubdb), (DRDAID = null), Begin compiling prepared statement: CREATE TABLE device (serial_number varchar(40), id_logical_device_type integer, tariff varchar(16), reference_power_active integer, overload_power_limit_active integer, breaker_state integer) :End prepared statement
Wed Oct 02 18:53:21 CEST 2013 Thread[main,5,main] (XID = 167), (SESSIONID = 1), (DATABASE = smarthubdb), (DRDAID = null), End compiling prepared statement: CREATE TABLE device (serial_number varchar(40), id_logical_device_type integer, tariff varchar(16), reference_power_active integer, overload_power_limit_active integer, breaker_state integer) :End prepared statement
Wed Oct 02 18:53:22 CEST 2013 Thread[main,5,main] (XID = 167), (SESSIONID = 1), (DATABASE = smarthubdb), (DRDAID = null), Executing prepared statement: CREATE TABLE device (serial_number varchar(40), id_logical_device_type integer, tariff varchar(16), reference_power_active integer, overload_power_limit_active integer, breaker_state integer) :End prepared statement
Wed Oct 02 18:53:23 CEST 2013 Thread[main,5,main] (XID = 167), (SESSIONID = 1), (DATABASE = smarthubdb), (DRDAID = null), Committing
Wed Oct 02 18:53:24 CEST 2013 Thread[main,5,main] (XID = 171), (SESSIONID = 1), (DATABASE = smarthubdb), (DRDAID = null), Begin compiling prepared statement: CREATE TABLE measure (id_device varchar(40) NOT NULL, date timestamp NOT NULL, value decimal(12,2), type integer, sent boolean) :End prepared statement
Wed Oct 02 18:53:24 CEST 2013 Thread[main,5,main] (XID = 171), (SESSIONID = 1), (DATABASE = smarthubdb), (DRDAID = null), End compiling prepared statement: CREATE TABLE measure (id_device varchar(40) NOT NULL, date timestamp NOT NULL, value decimal(12,2), type integer, sent boolean) :End prepared statement
Wed Oct 02 18:53:24 CEST 2013 Thread[main,5,main] (XID = 171), (SESSIONID = 1), (DATABASE = smarthubdb), (DRDAID = null), Executing prepared statement: CREATE TABLE measure (id_device varchar(40) NOT NULL, date timestamp NOT NULL, value decimal(12,2), type integer, sent boolean) :End prepared statement
Wed Oct 02 18:53:26 CEST 2013 Thread[main,5,main] (XID = 171), (SESSIONID = 1), (DATABASE = smarthubdb), (DRDAID = null), Committing
Wed Oct 02 18:53:27 CEST 2013 Thread[main,5,main] (XID = 184), (SESSIONID = 1), (DATABASE = smarthubdb), (DRDAID = null), Begin compiling prepared statement: CREATE TABLE message (id_device varchar(40) NOT NULL, date integer, message varchar(100)) :End prepared statement
Wed Oct 02 18:53:27 CEST 2013 Thread[main,5,main] (XID = 184), (SESSIONID = 1), (DATABASE = smarthubdb), (DRDAID = null), End compiling prepared statement: CREATE TABLE message (id_device varchar(40) NOT NULL, date integer, message varchar(100)) :End prepared statement
Wed Oct 02 18:53:27 CEST 2013 Thread[main,5,main] (XID = 184), (SESSIONID = 1), (DATABASE = smarthubdb), (DRDAID = null), Executing prepared statement: CREATE TABLE message (id_device varchar(40) NOT NULL, date integer, message varchar(100)) :End prepared statement
Wed Oct 02 18:53:27 CEST 2013 Thread[main,5,main] (XID = 184), (SESSIONID = 1), (DATABASE = smarthubdb), (DRDAID = null), Committing
Wed Oct 02 18:53:28 CEST 2013 Thread[main,5,main] (XID = 187), (SESSIONID = 1), (DATABASE = smarthubdb), (DRDAID = null), Begin compiling prepared statement: CREATE TABLE logical_device_type (id integer, name varchar(100)) :End prepared statement
Wed Oct 02 18:53:28 CEST 2013 Thread[main,5,main] (XID = 187), (SESSIONID = 1), (DATABASE = smarthubdb), (DRDAID = null), End compiling prepared statement: CREATE TABLE logical_device_type (id integer, name varchar(100)) :End prepared statement
Wed Oct 02 18:53:28 CEST 2013 Thread[main,5,main] (XID = 187), (SESSIONID = 1), (DATABASE = smarthubdb), (DRDAID = null), Executing prepared statement: CREATE TABLE logical_device_type (id integer, name varchar(100)) :End prepared statement
Wed Oct 02 18:53:28 CEST 2013 Thread[main,5,main] (XID = 187), (SESSIONID = 1), (DATABASE = smarthubdb), (DRDAID = null), Committing
Wed Oct 02 18:53:29 CEST 2013 Thread[main,5,main] (XID = 193), (SESSIONID = 1), (DATABASE = smarthubdb), (DRDAID = null), Begin compiling prepared statement: CREATE TABLE load_curve (id_device varchar(40) NOT NULL, date timestamp NOT NULL, value integer, error integer, sent boolean) :End prepared statement
Wed Oct 02 18:53:29 CEST 2013 Thread[main,5,main] (XID = 193), (SESSIONID = 1), (DATABASE = smarthubdb), (DRDAID = null), End compiling prepared statement: CREATE TABLE load_curve (id_device varchar(40) NOT NULL, date timestamp NOT NULL, value integer, error integer, sent boolean) :End prepared statement
Wed Oct 02 18:53:29 CEST 2013 Thread[main,5,main] (XID = 193), (SESSIONID = 1), (DATABASE = smarthubdb), (DRDAID = null), Executing prepared statement: CREATE TABLE load_curve (id_device varchar(40) NOT NULL, date timestamp NOT NULL, value integer, error integer, sent boolean) :End prepared statement
Wed Oct 02 18:53:29 CEST 2013 Thread[main,5,main] (XID = 193), (SESSIONID = 1), (DATABASE = smarthubdb), (DRDAID = null), Committing
Wed Oct 02 18:53:29 CEST 2013 Thread[main,5,main] (XID = 196), (SESSIONID = 1), (DATABASE = smarthubdb), (DRDAID = null), Begin compiling prepared statement: CREATE TABLE load_curve_last_collect (id_device varchar(40) NOT NULL, date timestamp NOT NULL) :End prepared statement
Wed Oct 02 18:53:30 CEST 2013 Thread[main,5,main] (XID = 196), (SESSIONID = 1), (DATABASE = smarthubdb), (DRDAID = null), End compiling prepared statement: CREATE TABLE load_curve_last_collect (id_device varchar(40) NOT NULL, date timestamp NOT NULL) :End prepared statement
Wed Oct 02 18:53:30 CEST 2013 Thread[main,5,main] (XID = 196), (SESSIONID = 1), (DATABASE = smarthubdb), (DRDAID = null), Executing prepared statement: CREATE TABLE load_curve_last_collect (id_device varchar(40) NOT NULL, date timestamp NOT NULL) :End prepared statement
Wed Oct 02 18:53:30 CEST 2013 Thread[main,5,main] (XID = 196), (SESSIONID = 1), (DATABASE = smarthubdb), (DRDAID = null), Committing
Wed Oct 02 18:53:30 CEST 2013 Thread[main,5,main] (XID = 199), (SESSIONID = 1), (DATABASE = smarthubdb), (DRDAID = null), Begin compiling prepared statement: CREATE TABLE tariff (type integer, name varchar(100)) :End prepared statement
Wed Oct 02 18:53:30 CEST 2013 Thread[main,5,main] (XID = 199), (SESSIONID = 1), (DATABASE = smarthubdb), (DRDAID = null), End compiling prepared statement: CREATE TABLE tariff (type integer, name varchar(100)) :End prepared statement
Wed Oct 02 18:53:30 CEST 2013 Thread[main,5,main] (XID = 199), (SESSIONID = 1), (DATABASE = smarthubdb), (DRDAID = null), Executing prepared statement: CREATE TABLE tariff (type integer, name varchar(100)) :End prepared statement
Wed Oct 02 18:53:31 CEST 2013 Thread[main,5,main] (XID = 199), (SESSIONID = 1), (DATABASE = smarthubdb), (DRDAID = null), Committing
Wed Oct 02 18:53:31 CEST 2013 Thread[main,5,main] (XID = 202), (SESSIONID = 1), (DATABASE = smarthubdb), (DRDAID = null), Begin compiling prepared statement: CREATE TABLE state_configuration (state_name varchar(50), last_execution timestamp) :End prepared statement
Wed Oct 02 18:53:31 CEST 2013 Thread[main,5,main] (XID = 202), (SESSIONID = 1), (DATABASE = smarthubdb), (DRDAID = null), End compiling prepared statement: CREATE TABLE state_configuration (state_name varchar(50), last_execution timestamp) :End prepared statement
Wed Oct 02 18:53:31 CEST 2013 Thread[main,5,main] (XID = 202), (SESSIONID = 1), (DATABASE = smarthubdb), (DRDAID = null), Executing prepared statement: CREATE TABLE state_configuration (state_name varchar(50), last_execution timestamp) :End prepared statement
Wed Oct 02 18:53:31 CEST 2013 Thread[main,5,main] (XID = 202), (SESSIONID = 1), (DATABASE = smarthubdb), (DRDAID = null), Committing
Wed Oct 02 18:53:32 CEST 2013 Thread[main,5,main] (XID = 205), (SESSIONID = 1), (DATABASE = smarthubdb), (DRDAID = null), Begin compiling prepared statement: CREATE TABLE metering_campaign_strategy (serial_number varchar(40) NOT NULL, measure_kind varchar(15) NOT NULL, frequencyUnit varchar(10) NOT NULL, frequency integer, begin_date timestamp NOT NULL, end_date timestamp, last_retrieve_date timestamp) :End prepared statement
Wed Oct 02 18:53:32 CEST 2013 Thread[main,5,main] (XID = 205), (SESSIONID = 1), (DATABASE = smarthubdb), (DRDAID = null), End compiling prepared statement: CREATE TABLE metering_campaign_strategy (serial_number varchar(40) NOT NULL, measure_kind varchar(15) NOT NULL, frequencyUnit varchar(10) NOT NULL, frequency integer, begin_date timestamp NOT NULL, end_date timestamp, last_retrieve_date timestamp) :End prepared statement
Wed Oct 02 18:53:32 CEST 2013 Thread[main,5,main] (XID = 205), (SESSIONID = 1), (DATABASE = smarthubdb), (DRDAID = null), Executing prepared statement: CREATE TABLE metering_campaign_strategy (serial_number varchar(40) NOT NULL, measure_kind varchar(15) NOT NULL, frequencyUnit varchar(10) NOT NULL, frequency integer, begin_date timestamp NOT NULL, end_date timestamp, last_retrieve_date timestamp) :End prepared statement
Wed Oct 02 18:53:33 CEST 2013 Thread[main,5,main] (XID = 205), (SESSIONID = 1), (DATABASE = smarthubdb), (DRDAID = null), Committing

谢谢

【问题讨论】：

你在使用连接池吗？你能从中得到一个陈旧的连接吗？
不，我们没有使用连接池。 Connection 对象持有与数据库的连接，用于获取执行查询的语句对象
这是整个跟踪吗？从跟踪开始到第一个数据库活动之间有 15 分钟的时间。如果您在应用程序启动后很快访问数据库会发生什么？
！是的，这是整个跟踪。我认为默认情况下 Derby 不会记录所有内容，我会更改它并看看结果如何！
如果您在启动应用程序后不久访问数据库会发生什么。我要去的地方是，它可能在应用程序初始化后的 15 分钟不活动中，连接或语句已经过时。

标签： java sql derby

【解决方案1】：

这是调试的一个挑战，因为在执行查询时在生成的代码中发生了崩溃。

对于您的查询，生成的代码可能正在评估限制谓词（WHERE serial_number = '041169700007' AND measure_kind='INDEX'）

首先，这里有一些关于如何了解生成代码内部内容的信息：http://wiki.apache.org/db-derby/DumpClassFile

问题似乎是间歇性发生的，这使得问题变得更加困难，因为如果没有可重现的案例，您将难以追踪和解决错误。

您的应用程序是单线程的吗？还是有多个线程在运行？

您是否检查过资源不足问题？ JVM内存耗尽、磁盘空间不足等？

可能值得考虑的一件事是 Derby 语句缓存是否会发挥作用。禁用缓存会影响问题吗？ http://wiki.apache.org/db-derby/StatementCache

您提到数据库架构（表、索引等）是动态创建的。您可以更新您的问题以包含确切的架构信息吗？一种方法是在运行应用程序后对数据库运行“dblook”。或者使用“-Dderby.language.logStatementText=true”运行您的应用程序，所有执行的 SQL 语句都应回显到日志中。

如果您自己没有走得太远，您可以尝试联系 Apache 邮件列表中的 Derby 社区，因为他们会有更多建议供您尝试。

【讨论】：

应用程序是单线程的。确保有足够的磁盘空间，就内存而言，我没有收到任何此类消息（在某些情况下，我们会出现 30 分钟的异常，然后一切都会恢复正常一段时间一两个小时）。我将尝试使用更多的堆/堆栈来运行 cvm。我将更新我的帖子以包含更多信息
刚刚编辑了我的问题以添加您要求的信息。如果我发现任何有趣的东西，我会查看语句缓存并发布

【解决方案2】：

问题只是你有一些数据是NULL，由于某些原因你没有确定某些时间和NOT NULL其他时间。

如果此数据来自数据库，则将所有列设置为 NOT NULLABLE 将暴露 NULL 数据的插入位置。

如果数据来自您的模型代码，那么您可以使用来自 Google 的 JSR 305 库在您的所有方法签名中使用 @Nonnull 注释所有内容，您会发现 NULL 引用被传递的位置。

从长远来看，这两件事都会提高代码的可维护性和质量。

【讨论】：