【问题标题】:Recurring tasks using SSIS使用 SSIS 的重复任务
【发布时间】:2018-06-20 22:22:57
【问题描述】:

我创建了一个新的 SSIS 包 (SQL 2017),它读取“导出请求”表并根据计划将数据导出到外部存储。例如,用户可以向该表添加一个新行,如“select * from table 1 where col1=2”,时间表为 2018 年 6 月 25 日下午 3:30。

我的包(使用 SQL 代理运行)一直循环以在该表中搜索新任务,并在计划时将查询结果导出到当时的文件夹。

有一项新功能要求将这些导出安排为重复出现。例如,客户可能希望从 2018 年 6 月 25 日下午 3:30 开始每天进行导出。每年、每月、每天、每 3 个月的日程安排可能会变得复杂......

在 SSIS 中实现循环的最佳方式是什么?我可以为 Cron 使用 5 个字符的字段,但不知道如何使用他们的 Cron 计划查询新任务。 有什么帮助吗?

【问题讨论】:

  • 为什么不能使用SQL Agent的调度能力?

标签: sql ssis cron


【解决方案1】:

确认您已经拥有的内容:您有一个 TASK 表,其中包含任务以及需要运行的某种日期标志。然后,您的作业每 x 分钟运行一次,并检查是否有任何任务要执行/运行。如果是这样,它会运行它们。然后我猜你在你设置的表中有一个标志,说它在你在循环中运行后完成?如果您一次要运行多个任务,它一次只运行一个?

然后你想要做的就是添加这个:我做了一些非常相似的事情。我为每个任务创建了一个查找表(在我的流程中,每个任务都有一个名称,我使用该名称来引用查找表,我称之为调度查找)。

在该表中,我列出了报告需要运行的时间和频率(所以 2 列)一天中的时间和频率。一个例子是 7:00 和工作日。因此,对于此报告,它仅在每个工作日 (m-F) 和晚上 7 点运行。

然后,当我的作业运行时,它会像您上面那样运行任务,但随后会有另一个步骤,将该任务标记为已完成,然后在任务表中插入一个新任务(任务详细信息将相同) 但我会查看上面描述的日程表查找表,以确定作业应该再次运行的下一个日期/时间,并将其用作我的任务表中的下一个运行日期/时间。

以下是我在流程中使用的 SP,用于获取下一个计划日期/时间并将现有的更新为已完成,然后创建新的。

注意:我的日程安排有一些您可能不需要的高级选项,我有很多 cmets 应该解释我在做什么/为什么要做所有事情。我正在调用我在某些地方创建的一些函数,但我认为您不需要这些函数,并且可以弄清楚要做什么而不是我的函数,但是如果您有问题,请告诉我:

这是我使用的,所以它使用我的表结构/等,但你可以很容易地将它转换为你的。

--Purpose
----------------------------------------------------------------------------------
-- calculates the next time to run/schedule the job
----------------------------------------------------------------------------------
    -- NOTE:  TO USE you have to insert the first value manually in queue table

    ----- possibile scenerios
    -- if we want to schedule every x hours, or x days
    -- run every month only
    -- run weekdays only
    -- run on certain days of month only


    -- TO ADD MORE COMPLEX or different types of schedules:
    --  special - different times for different days of week
    --          ex - so have dayofweek:2:00,dayofweek:3:00  (and we parse out the day of week and number splitting out the strings)

    --  hourly - to do more then once a day??
    --          WHEN @ScheduleLookupType = 'hourly' THEN DATEADD(DAY, 1, @CurrentScheduleDate) -- FIX FIX FIX


--  EXEC dbo.JobsDynamicRescheduleFindNextTimeToScheduleJob @ReportName = 'TestReport1'

----------------------------------------------------------------------------------


ALTER PROCEDURE [dbo].[JobsDynamicRescheduleFindNextTimeToScheduleJob]
    @ReportName VARCHAR(50)
AS

SET NOCOUNT ON

BEGIN TRY
            -- left here for testing outside of SP
            -- this will be passed from SP
            --DECLARE @ReportName AS VARCHAR(50)    
            --SET @ReportName = 'TESTREport'


            -- this sets the first day of the week to Monday (we need it set to a value to do calcluations for weekdays, I set it to 1 for monday and 7 for sunday)
            --  this is due to server settings could have somethign else so forcing it here
            SET DATEFIRST 1

            DECLARE @CurrentScheduleDate AS DATE                    -- find the current date for the job that just ran
            DECLARE @CurrentScheduleDayNumberOfWeek AS SMALLINT     -- this pulls the number of the day of week 1=monday, 2=tuesdday
            DECLARE @ScheduleLookupType AS VARCHAR(20)              -- this is the type of schedule to do calculations on
            DECLARE @TimeOfDayToScheduleJob AS VARCHAR(20)          -- look this up, its the time to schedule the job
            DECLARE @SpecialScheduleValue AS VARCHAR(8000)          -- this is special value to lookup (only needed if non standard one)
            DECLARE @NewScheduleDateONLY AS DATETIME                -- to hold just the date of the schedule before combinng with time
            DECLARE @NewScheduleDateTime AS DATETIME                -- to hold the new schedule date and time, actual value to insert into queue


            -- pull the current schedule date/time from the queue table
            SELECT @CurrentScheduleDate = NextRunDateTime
            FROM dbo.GenericReportingQueue (NOLOCK)
            WHERE IsGenerated IS NULL
            AND ReportName = @ReportName


            -- to override for testing
            --SET @CurrentScheduleDate = '5/20/2016'


            SET @CurrentScheduleDayNumberOfWeek = DATEPART(WEEKDAY, @CurrentScheduleDate)

            -- pull these values from lookup table
            SELECT @ScheduleLookupType = dbo.fn_GetValueLookupTableValue(@ReportName, 'RescheduleJobDynamic_ScheduleLookupType'), 
            @TimeOfDayToScheduleJob = dbo.fn_GetValueLookupTableValue(@ReportName, 'RescheduleJobDynamic_TimeOfDayToScheduleJob'),
            @SpecialScheduleValue = dbo.fn_GetValueLookupTableValue(@ReportName, 'RescheduleJobDynamic_SpecialScheduleValue')


            /*
            -- reset for testing
            SET @ScheduleLookupType = 'specialdays' -- weekly, weekdays, monthly, specialdays
            SET @TimeOfDayToScheduleJob = '8:00'
            SET @SpecialScheduleValue = '5,6'
            */

            -- calculations to get the date to schedule the job next time based off logic
            SELECT @NewScheduleDateONLY = CASE  
                WHEN @ScheduleLookupType = 'daily' THEN DATEADD(DAY, 1, @CurrentScheduleDate)
                WHEN @ScheduleLookupType = 'weekly' THEN DATEADD(DAY, 7, @CurrentScheduleDate)
                WHEN @ScheduleLookupType = 'monthly' THEN DATEADD(MONTH, 1, @CurrentScheduleDate)
                WHEN @ScheduleLookupType = 'yearly' THEN DATEADD(YEAR, 1, @CurrentScheduleDate)

                -- only run on weekdays and skip weekends
                WHEN @ScheduleLookupType = 'weekdays' THEN 
                    CASE 
                        WHEN @CurrentScheduleDayNumberOfWeek IN (1, 2, 3, 4) THEN DATEADD(DAY, 1, @CurrentScheduleDate)
                        WHEN @CurrentScheduleDayNumberOfWeek = 5 THEN DATEADD(DAY, 3, @CurrentScheduleDate)
                    END -- end case for day of week
                -- only run on weekends and skip weekdays
                WHEN @ScheduleLookupType = 'weekends' THEN 
                    CASE
                        WHEN @CurrentScheduleDayNumberOfWeek = 6 THEN DATEADD(DAY, 1, @CurrentScheduleDate)
                        WHEN @CurrentScheduleDayNumberOfWeek = 7 THEN DATEADD(DAY, 6, @CurrentScheduleDate)
                    END -- end case for weekends only

                WHEN @ScheduleLookupType = 'specialdays' THEN
                    -- for this we need to determine the current day, and the next day we want to run on, then add that many days
                    --      if next day is not till the following week we just find the first day in the list
                    --      Take taht number and do dateadd to it
                    DATEADD(DAY, 
                            -- this does the select to determine what number to add based off current day and next day list
                            (SELECT ISNULL(
                                -- if this one I want to take today value and subtract from next value found
                                --      then add that number to todays date to give me the next schedule date
                                (SELECT TOP 1 StringValue - @CurrentScheduleDayNumberOfWeek
                                FROM dbo.fn_ParseText2Table(@SpecialScheduleValue, ',')
                                WHERE StringValue > @CurrentScheduleDayNumberOfWeek
                                ORDER BY StringValue)
                                ,
                                -- if none found above I need to go to the next weeks first value
                                --      I need to take 7 - todays number (to get the rest of the week) then add the next number for the next week to it
                                (SELECT TOP 1 (7 - @CurrentScheduleDayNumberOfWeek) + StringValue 
                                FROM dbo.fn_ParseText2Table(@SpecialScheduleValue, ',')                 
                                ORDER BY StringValue)
                                )-- end is null
                            ) -- end select 
                        , @CurrentScheduleDate) -- end dateadd for speical days             
            END -- outer case


            SET @NewScheduleDateTime = @NewScheduleDateONLY + ' ' + @TimeOfDayToScheduleJob

            -- for testing
            --SELECT @ScheduleLookupType AS ReportLookupType, @TimeOfDayToScheduleJob AS TimeOfDayToSchedule, @SpecialScheduleValue AS SpecialValuesForCalc, @NewScheduleDateTime AS NewDateTimeToRun, 
            --@CurrentScheduleDate AS CurrentDateSchedule, @CurrentScheduleDayNumberOfWeek AS CurrentNumberDayOfWeek, @NewScheduleDateONLY AS NewScheduleDateOnly


            -- &&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&
            --      now update and insert the new schedule date/time into the table

            -- update existing record
            UPDATE dbo.GenericReportingQueue
            SET IsGenerated = 1,
            DateReportRun = GETDATE(),
            LastUpdateDate = GETDATE()
            WHERE ISGenerated IS NULL
            AND ReportName = @ReportName


            -- insert new record with new date
            INSERT INTO dbo.GenericReportingQueue (
                ReportName, NextRunDateTime, CreatorID, CreateDate
            )
            SELECT @ReportName, @NewScheduleDateTime, 1, GETDATE()



END TRY

BEGIN CATCH

   RETURN

END CATCH

【讨论】:

  • 谢谢布拉德。这正是我一直在寻找的。欣赏它。
猜你喜欢
  • 2021-12-02
  • 2011-04-07
  • 1970-01-01
  • 1970-01-01
  • 1970-01-01
  • 1970-01-01
  • 1970-01-01
  • 1970-01-01
  • 2013-04-05
相关资源
最近更新 更多