Quartz.Net，SQL Server，作业卡在 BLOCKED

Question

我到处搜索，没有发现对我有用的东西。

我有 Quartz.Net 2.5。我使用集群模式并使用 SQL Server 持久化我的作业。

有时我会遇到这样的异常，我的工作卡在BLOCKED 状态。

如何更正配置或更改代码以解决作业停止的问题？

2017-08-04 00:53:17,981 (47972313) [iikoNetScheduler_QuartzSchedulerThread] ERROR - Couldn't rollback ADO.NET connection. Transaction not connected, or was disconnected
System.Data.DataException: Transaction not connected, or was disconnected
   at Quartz.Impl.AdoJobStore.JobStoreSupport.CheckNotZombied(ConnectionAndTransactionHolder cth) in c:\projects\quartznet\src\Quartz\Impl\AdoJobStore\JobStoreSupport.cs:line 3591
   at Quartz.Impl.AdoJobStore.JobStoreSupport.RollbackConnection(ConnectionAndTransactionHolder cth) in c:\projects\quartznet\src\Quartz\Impl\AdoJobStore\JobStoreSupport.cs:line 3364
2017-08-04 00:53:17,982 (47972314) [iikoNetScheduler_QuartzSchedulerThread] ERROR - An error occurred while firing triggers 'System.Collections.Generic.List`1[Quartz.Spi.IOperableTrigger]'
Quartz.JobPersistenceException: Couldn't commit ADO.NET transaction. Timeout expired.  The timeout period elapsed prior to completion of the operation or the server is not responding. ---> System.Data.SqlClient.SqlException: Timeout expired.  The timeout period elapsed prior to completion of the operation or the server is not responding. ---> System.ComponentModel.Win32Exception: The wait operation timed out
   --- End of inner exception stack trace ---
   at System.Data.SqlClient.SqlConnection.OnError(SqlException exception, Boolean breakConnection, Action`1 wrapCloseInAction)
   at System.Data.SqlClient.TdsParser.ThrowExceptionAndWarning(TdsParserStateObject stateObj, Boolean callerHasConnectionLock, Boolean asyncClose)
   at System.Data.SqlClient.TdsParser.TryRun(RunBehavior runBehavior, SqlCommand cmdHandler, SqlDataReader dataStream, BulkCopySimpleResultSet bulkCopyHandler, TdsParserStateObject stateObj, Boolean& dataReady)
   at System.Data.SqlClient.TdsParser.Run(RunBehavior runBehavior, SqlCommand cmdHandler, SqlDataReader dataStream, BulkCopySimpleResultSet bulkCopyHandler, TdsParserStateObject stateObj)
   at System.Data.SqlClient.TdsParser.TdsExecuteTransactionManagerRequest(Byte[] buffer, TransactionManagerRequestType request, String transactionName, TransactionManagerIsolationLevel isoLevel, Int32 timeout, SqlInternalTransaction transaction, TdsParserStateObject stateObj, Boolean isDelegateControlRequest)
   at System.Data.SqlClient.SqlInternalConnectionTds.ExecuteTransactionYukon(TransactionRequest transactionRequest, String transactionName, IsolationLevel iso, SqlInternalTransaction internalTransaction, Boolean isDelegateControlRequest)
   at System.Data.SqlClient.SqlInternalTransaction.Commit()
   at System.Data.SqlClient.SqlTransaction.Commit()
   at Quartz.Impl.AdoJobStore.JobStoreSupport.CommitConnection(ConnectionAndTransactionHolder cth, Boolean openNewTransaction) in c:\projects\quartznet\src\Quartz\Impl\AdoJobStore\JobStoreSupport.cs:line 3395
   --- End of inner exception stack trace ---
   at Quartz.Impl.AdoJobStore.JobStoreSupport.CommitConnection(ConnectionAndTransactionHolder cth, Boolean openNewTransaction) in c:\projects\quartznet\src\Quartz\Impl\AdoJobStore\JobStoreSupport.cs:line 3403
   at Quartz.Impl.AdoJobStore.JobStoreSupport.ExecuteInNonManagedTXLock[T](String lockName, Func`2 txCallback, Func`3 txValidator) in c:\projects\quartznet\src\Quartz\Impl\AdoJobStore\JobStoreSupport.cs:line 3562
   at Quartz.Impl.AdoJobStore.JobStoreSupport.TriggersFired(IList`1 triggers) in c:\projects\quartznet\src\Quartz\Impl\AdoJobStore\JobStoreSupport.cs:line 2592
   at Quartz.Core.QuartzSchedulerThread.Run() in c:\projects\quartznet\src\Quartz\Core\QuartzSchedulerThread.cs:line 381 [See nested exception: System.Data.SqlClient.SqlException (0x80131904): Timeout expired.  The timeout period elapsed prior to completion of the operation or the server is not responding. ---> System.ComponentModel.Win32Exception (0x80004005): The wait operation timed out
   at System.Data.SqlClient.SqlConnection.OnError(SqlException exception, Boolean breakConnection, Action`1 wrapCloseInAction)
   at System.Data.SqlClient.TdsParser.ThrowExceptionAndWarning(TdsParserStateObject stateObj, Boolean callerHasConnectionLock, Boolean asyncClose)
   at System.Data.SqlClient.TdsParser.TryRun(RunBehavior runBehavior, SqlCommand cmdHandler, SqlDataReader dataStream, BulkCopySimpleResultSet bulkCopyHandler, TdsParserStateObject stateObj, Boolean& dataReady)
   at System.Data.SqlClient.TdsParser.Run(RunBehavior runBehavior, SqlCommand cmdHandler, SqlDataReader dataStream, BulkCopySimpleResultSet bulkCopyHandler, TdsParserStateObject stateObj)
   at System.Data.SqlClient.TdsParser.TdsExecuteTransactionManagerRequest(Byte[] buffer, TransactionManagerRequestType request, String transactionName, TransactionManagerIsolationLevel isoLevel, Int32 timeout, SqlInternalTransaction transaction, TdsParserStateObject stateObj, Boolean isDelegateControlRequest)
   at System.Data.SqlClient.SqlInternalConnectionTds.ExecuteTransactionYukon(TransactionRequest transactionRequest, String transactionName, IsolationLevel iso, SqlInternalTransaction internalTransaction, Boolean isDelegateControlRequest)
   at System.Data.SqlClient.SqlInternalTransaction.Commit()
   at System.Data.SqlClient.SqlTransaction.Commit()
   at Quartz.Impl.AdoJobStore.JobStoreSupport.CommitConnection(ConnectionAndTransactionHolder cth, Boolean openNewTransaction) in c:\projects\quartznet\src\Quartz\Impl\AdoJobStore\JobStoreSupport.cs:line 3395
ClientConnectionId:ac0017d3-abf7-472d-bf8f-753fbeb000be
Error Number:-2,State:0,Class:11]

所以在那之后我的工作挂在了 BLOCKED 状态，此时没有执行工作。 这让我抓狂，为什么石英在失去连接后会停止我的工作？ 我希望 Quartz 能正确处理这种情况。

StdSchedulerFactory 的 MySettings：

            // configure Thread Pool
            properties["quartz.threadPool.type"] = "Quartz.Simpl.SimpleThreadPool, Quartz";
            properties["quartz.threadPool.makeThreadsDaemons"] = "true";
            properties["quartz.threadPool.threadCount"] = "5";
            properties["quartz.threadPool.threadPriority"] = "Normal";

            properties["quartz.scheduler.instanceName"] = schedulerInfo.Name;
            properties[StdSchedulerFactory.PropertySchedulerInterruptJobsOnShutdown] = "true";
            properties[StdSchedulerFactory.PropertySchedulerMakeSchedulerThreadDaemon] = "true;";
            // configure Job Store
            properties["quartz.jobStore.misfireThreshold"] = "60000";
                properties["quartz.jobStore.type"] = "Quartz.Impl.AdoJobStore.JobStoreTX, Quartz";
                properties["quartz.jobStore.driverDelegateType"] = "Quartz.Impl.AdoJobStore.SqlServerDelegate, Quartz";
                // all values in JobDataMaps will be Strings, and therefore can be stored as name-value pairs
                properties["quartz.jobStore.useProperties"] = "true";
                properties["quartz.jobStore.tablePrefix"] = "QRTZ_";
                properties["quartz.jobStore.clustered"] = "true";
                properties["quartz.scheduler.instanceId"] = "AUTO";
                properties["quartz.scheduler.dbFailureRetryInterval"] = "60000";
                properties["quartz.jobStore.dataSource"] = "default";
                properties["quartz.dataSource.default.provider"] = "SqlServer-20";
                properties["quartz.dataSource.default.connectionString"] = schedulerInfo.ConnectionString;
                properties["quartz.dataSource.default.maxConnections"] = "10";

我这样创建工作：

var jobDetail =     JobBuilder.Create(jobTemplate.JobType)
                            .WithIdentity(jobTemplate.JobName, jobTemplate.GroupName)
                            .RequestRecovery(true)
                            .StoreDurably(true)
                            .Build();
        Scheduler.AddJob(jobDetail, true);
        var tb = TriggerBuilder.Create()
    .WithIdentity(jobTemplate.TriggerName, jobTemplate.GroupName)
    .ForJob(jobDetail);
    tb.WithSimpleSchedule(x => FillDefaultSimpleScheduleBuilder(x, intervalJobTemplate.Interval, intervalJobTemplate.RepeatCount));
        trigger = tb.Build();
        Scheduler.ScheduleJob(trigger);

帮助方法：

private static void FillDefaultSimpleScheduleBuilder(SimpleScheduleBuilder inputBuilder, int interval, int? repeatCount)
    {
        var builder = inputBuilder.WithIntervalInSeconds(interval);
        builder = repeatCount.HasValue ? builder.WithRepeatCount(repeatCount.Value) : builder.RepeatForever();

        // fire once immediately if misfire occurs.
        builder.WithMisfireHandlingInstructionNowWithExistingCount();
    }

Answer 1

尝试扩大石英 SQL 命令的命令超时时间。 为此，请定义您自己的 SqlServerDelegate：

namespace QuartzHelper
{
    /// <summary>
    /// Wrapper around <see cref="SqlServerDelegate"/>, adjusting default command timeout.
    /// </summary>
    internal class LongTimeoutSqlServerDelegate : SqlServerDelegate
    {
        private readonly int quartzCommandTimeoutSeconds = 100;

        public override System.Data.IDbCommand PrepareCommand(ConnectionAndTransactionHolder cth, string commandText)
        {
            var command = base.PrepareCommand(cth, commandText);
            command.CommandTimeout = quartzCommandTimeoutSeconds;
            return command;
        }
    }
}

...并在创建 StdSchedulerFactory 时使用它而不是默认 "Quartz.Impl.AdoJobStore.SqlServerDelegate, Quartz":

properties["quartz.jobStore.driverDelegateType"] = "QuartzHelper.LongTimeoutSqlServerDelegate";