【问题标题】:Get ROWCOUNT for all actions performed in MERGE Statement获取在 MERGE 语句中执行的所有操作的 ROWCOUNT
【发布时间】:2019-06-23 01:58:53
【问题描述】:

我对这个 MERGE 声明感到惊讶,该公司并没有真正做到 Type 2 Slowing Change Dimension,而是接近了。奇怪的是,它甚至不是分析数据,但让我们忽略这个可怕的决定。我有这个工作引用HashBytes 来指示更改的行。不幸的是,为了解决所有场景,我最终从实际保存更新行的临时表中获得了额外的INSERT

唉,它很实用,但如果你有更有效的设计,请分享。我会很感激。

但是,我试图从Temp 表中获取一个row count,不仅代表INSERT,而且更新和新的INSERTS,都是具有自己row count 的不同的独立操作,我需要记录和说明。

请问我该怎么做?

DECLARE @dtNow AS DATETIME = GetDate()
DECLARE @dtPast AS DATETIME = DATEADD(day,-1,GetDate())
DECLARE @dtFuture AS DATETIME = '22991231'
    SET NOCOUNT ON;

  -- Temp Table is JUST Updating Rows reflecting 
--Historical Marker on existing row No content change to row's columnar     content data 

IF OBJECT_ID('tempdb..#TheTempTableName') IS NOT NULL DROP TABLE     #TheTempTableName


CREATE TABLE #TheTempTableName 
(
    ABunchOfColumns
    RowCreatedDate datetime NULL,
    RowEffectiveDate datetime NULL,
    RowTerminationDate datetime NULL,
    RowIsCurrent bit NULL,
    RowHash varchar(max) NULL,
)

INSERT INTO #TheTempTableName
(
            ABunchOfColumns
                ,RowCreatedDate
                ,RowEffectiveDate
                ,RowTerminationDate
                ,RowIsCurrent
                ,RowHash 
)


SELECT
                ABunchOfColumns
                ,RowCreatedDate
                ,RowEffectiveDate
                ,RowTerminationDate
                ,RowIsCurrent
                ,RowHash 
FROM
    (
        MERGE tblDim WITH (HOLDLOCK) AS target
            USING 
            (
                SELECT          
                ABunchOfColumns
                ,RowCreatedDate
                ,RowEffectiveDate
                ,RowTerminationDate
                ,RowIsCurrent
                ,RowHash 

                FROM dbo.tblStaging
            )       
        AS source
        ON target.PKID  = source.PKID       
        WHEN MATCHED
            AND target.RowIsCurrent = 1
            AND target.RowHash != source.RowHash                    
------- PROCESS ONE -- UPDATE --- HISTORICALLY MARK EXISTING ROWS
            THEN UPDATE SET                     
                    RowEffectiveDate    =   @dtPast
                    ,RowTerminationDate =   @dtPast
                    ,RowIsCurrent   =   0                   
-----  PROCESS TWO -- INSERT ---INSERT NEW ROWS
            WHEN NOT     MATCHED                                                          
            THEN INSERT    ---   THIS INSERT Goes directly into Target ( DIM     ) Table (New Rows not matched with PK = PK ) 
                (
                ABunchOfColumns
                ,RowCreatedDate
                ,RowEffectiveDate
                ,RowTerminationDate
                ,RowIsCurrent
                ,RowHash 
                )
                VALUES
                (                   
                source.ABunchOfColumns
                    ,@dtNow                 --source.RowCreatedDate,
                    ,@dtFuture              ---source.RowEffectiveDate,
                    ,@dtFuture              ---source.RowTerminationDate,
                    ,1                      ---source.RowIsCurrent,
                    ,source.RowHash
                )
-------PROCESS THREE a  -- INSERT ---OUTPUT MATCHED ROWS FROM PROCESS ONE THAT CAUSED     HISTORICAL MARK (CHANGES) "INSERT"
        OUTPUT 
        $action Action_Out,

                ABunchOfColumns
                ,RowCreatedDate
                ,RowEffectiveDate
                ,RowTerminationDate
                ,RowIsCurrent
                ,RowHash 
    )       
    AS MERGE_OUT
WHERE MERGE_OUT.Action_Out = 'UPDATE';

----------PROCESS THREE b  -- INSERT FROM Temp Tbl to final
--Now we flush the data in the temp table into dim table

INSERT INTO tblDim
    (
                ABunchOfColumns
                ,RowCreatedDate
                ,RowEffectiveDate
                ,RowTerminationDate
                ,RowIsCurrent
                ,RowHash 
    )
SELECT
         ABunchOfColumns
        ,@dtNow AS RowCreatedDate
        ,@dtFuture AS RowEffectiveDate
        ,@dtFuture AS RowTerminationDate
        ,1 AS RowIsCurrent
        ,RowHash

FROM #TheTempTableName  
END

【问题讨论】:

  • 希望您知道,您可以使用更简单的代码和单独的 INSERT/UPDATE 语句来做同样的事情。我个人避免使用 MERGE,这是一个原因:mssqltips.com/sqlservertip/3074/… 最后请注意,任何散列算法都可能发生冲突(两组不同的数据导致相同的散列)。因此,在回答“更有效的设计”时,我建议单独插入/更新,但这实际上只是关于是否更有效的意见

标签: sql sql-server merge ssis etl


【解决方案1】:

根据您进一步的 cmets,我认为主要问题是您如何处理类型 2 更新。快速的回答是你需要两次更新操作(插入/更新);和 DELETES 并不是真正的 DELETES,而是时间戳上的 UPDATES。

我在如何处理 type2 更新下面制定了一个示例查询,结果应该是不言自明的。我曾尝试对 UPDATE 合并执行双重操作,但有趣的是它无法执行此操作并给出错误:“在MERGE 语句。” 所以我觉得别无选择,只能拆分 UPDATE 语句的更新和插入。

最后一个考虑因素也是作为更新体现的 DELETE。我已经在下面的代码中处理了如何确定 UPDATE 操作何时真正是 DELETE。

DROP TABLE IF EXISTS _a
CREATE TABLE _a (
    id int
,val int
,fromdate datetime
,todate datetime
,isactive bit
)

INSERT INTO _a
select 1,100,'2015-Jan-1',NULL,1
UNION ALL select 2,200,'2015-Feb-1',NULL,1
UNION ALL select 3,300,'2015-Mar-1',NULL,1

DROP TABLE IF EXISTS #data
DROP TABLE IF EXISTS #outputdata

select * INTO #data from _a
select TOP 0 action=CAST('' as varchar(10)),* INTO #outputdata from _a

DELETE #data where id = 3
UPDATE #data set val = 2000 where id = 2
INSERT INTO #data
select 4,400,GETDATE(),NULL,1

--select * from #data

-- _a is your data warehouse table using type2

BEGIN TRAN

select Note='OLD STATE OF _a',* from _a

select Note='NEW SET OF DATA',* from #data


MERGE dbo._a T
USING (
    select id,val from #data
) S
ON (S.id = T.id)
WHEN MATCHED
AND ((S.val <> T.val OR (S.val IS NOT NULL AND T.val IS NULL) OR (S.val IS NULL AND T.val IS NOT NULL)))
THEN UPDATE SET  
        todate   = GETDATE()
    ,isactive = 0
WHEN NOT MATCHED BY TARGET
THEN INSERT (id,val,fromdate,todate,isactive)
        VALUES (id,val,GETDATE(),NULL,1)
WHEN NOT MATCHED BY SOURCE --AND T.id IN (SELECT id FROM #data)
--THEN DELETE TYPE2
THEN UPDATE SET  /*NO-PK*/ 
        todate   = GETDATE()
    ,isactive = 0
OUTPUT $action as Action
    ,ISNULL(inserted.id,deleted.id)  as id
    ,ISNULL(inserted.val,deleted.val)  as val
    ,ISNULL(inserted.fromdate,deleted.fromdate)  as fromdate
    ,ISNULL(inserted.todate,deleted.todate)  as todate
    ,ISNULL(inserted.isactive,deleted.isactive)  as isactive
INTO #outputdata;

select Note='Logs Output',* from #outputdata

-- FIND THE NEW RECORD
INSERT INTO _a (id,val,fromdate,todate,isactive)
SELECT a.id,a.val,GETDATE()+.000001,a.todate,a.isactive
FROM #data a
INNER JOIN #outputdata b
on a.id = b.id
WHERE b.action ='UPDATE'

select Note='NEW STATE OF _a',* from _a

SELECT Note='Real Action',d1.id,action=CASE WHEN action='UPDATE' AND d2.id is null then 'DELETE' ELSE action END
FROM #outputdata d1
LEFT JOIN _a d2
on d1.action ='UPDATE' and d1.id = d2.id and d2.isactive =1

ROLLBACK TRAN

结果将是:

+-----------------+----+-----+-------------------------+--------+----------+
|      Note       | id | val |        fromdate         | todate | isactive |
+-----------------+----+-----+-------------------------+--------+----------+
| OLD STATE OF _a |  1 | 100 | 2015-01-01 00:00:00.000 | NULL   |        1 |
| OLD STATE OF _a |  2 | 200 | 2015-02-01 00:00:00.000 | NULL   |        1 |
| OLD STATE OF _a |  3 | 300 | 2015-03-01 00:00:00.000 | NULL   |        1 |
+-----------------+----+-----+-------------------------+--------+----------+

+-----------------+----+------+-------------------------+--------+----------+
|      Note       | id | val  |        fromdate         | todate | isactive |
+-----------------+----+------+-------------------------+--------+----------+
| NEW SET OF DATA |  1 |  100 | 2015-01-01 00:00:00.000 | NULL   |        1 |
| NEW SET OF DATA |  2 | 2000 | 2015-02-01 00:00:00.000 | NULL   |        1 |
| NEW SET OF DATA |  4 |  400 | 2019-01-31 09:49:45.943 | NULL   |        1 |
+-----------------+----+------+-------------------------+--------+----------+

+-------------+--------+----+-----+-------------------------+-------------------------+----------+
|    Note     | action | id | val |        fromdate         |         todate          | isactive |
+-------------+--------+----+-----+-------------------------+-------------------------+----------+
| Logs Output | INSERT |  4 | 400 | 2019-01-31 09:51:13.647 | NULL                    |        1 |
| Logs Output | UPDATE |  2 | 200 | 2015-02-01 00:00:00.000 | 2019-01-31 09:51:13.647 |        0 |
| Logs Output | UPDATE |  3 | 300 | 2015-03-01 00:00:00.000 | 2019-01-31 09:51:13.647 |        0 |
+-------------+--------+----+-----+-------------------------+-------------------------+----------+

-- OPERATIONS 1 INSERT 1 UPDATE 1 DELETE
DELETE #data where id = 3
UPDATE #data set val = 2000 where id = 2
INSERT INTO #data
select 4,400,GETDATE(),NULL,1

+-----------------+----+------+-------------------------+-------------------------+----------+
|      Note       | id | val  |        fromdate         |         todate          | isactive |
+-----------------+----+------+-------------------------+-------------------------+----------+
| NEW STATE OF _a |  1 |  100 | 2015-01-01 00:00:00.000 | NULL                    |        1 |
| NEW STATE OF _a |  2 |  200 | 2015-02-01 00:00:00.000 | 2019-01-31 09:51:13.647 |        0 |
| NEW STATE OF _a |  3 |  300 | 2015-03-01 00:00:00.000 | 2019-01-31 09:51:13.647 |        0 |
| NEW STATE OF _a |  4 |  400 | 2019-01-31 09:51:13.647 | NULL                    |        1 |
| NEW STATE OF _a |  2 | 2000 | 2019-01-31 09:51:13.733 | NULL                    |        1 |
+-----------------+----+------+-------------------------+-------------------------+----------+

+-------------+----+--------+
|    Note     | id | action |
+-------------+----+--------+
| Real Action |  4 | INSERT |
| Real Action |  2 | UPDATE |
| Real Action |  3 | DELETE |
+-------------+----+--------+

【讨论】:

    【解决方案2】:

    有两种类型的删除 (1) 真正的删除 (2) 主键更新。 所以你也可以说有两种类型的插入(1)真正的插入(2)主键更新 更新总是更新。

    然后的困境是插入/删除组合何时才是真正的更新。

    通常如果你真的不关心上面的一个简单的合并就足够了

    MERGE esqlProductTarget T
    USING esqlProductSource S
    ON (S.ProductID = T.ProductID)
    WHEN MATCHED
    THEN UPDATE
         SET    T.Name = S.Name,
                T.ProductNumber = S.ProductNumber,
                T.Color = S.Color
    WHEN NOT MATCHED BY TARGET
    THEN INSERT (ProductID, Name, ProductNumber, Color)
         VALUES (S.ProductID, S.Name, S.ProductNumber, S.Color)
    WHEN NOT MATCHED BY SOURCE
    THEN DELETE
    OUTPUT S.ProductID, $action into @MergeLog;
    
    SELECT MergeAction, Cnt=count(*)
    FROM   @MergeLog
    GROUP BY MergeAction
    

    输出如下:

    +-------------+-----+--+
    | MergeAction | Cnt |  |
    +-------------+-----+--+
    | DELETE      | 100 |  |
    | UPDATE      |  60 |  |
    | INSERT      |  70 |  |
    +-------------+-----+--+
    

    参考https://www.essentialsql.com/introduction-merge-statement/

    我不确定你为什么有“WHERE MERGE_OUT.Action_Out = 'UPDATE'。但如果你删除它,那么你可以获得你的行数。除非我误解了你的查询。

    【讨论】:

    • 感谢您的洞察力和时间。在这种情况下,DELETE 是退休,更新行以退休。这意味着更新(退休)现有的和插入新的。我的 TEMP tbl 包含必须插入到 DIM 表的行。目标是在该 TEMP 表中容纳和识别 ($action) 执行的所有操作。我需要在 MERGE 语句指定适当的 ($action) 行后插入 INSERT 最后按 OPERATION TYPE 计算 TEMP 表的内容以保存管理员记录。也就是说,如果我能弄清楚如何在 Temp 表中计算所有操作
    • 谢谢“Earl”注意我的临时表“#TheTempTableName”。我正在使用它来收集在 MERGE 完成后插入到“DIM”表中的行。我是否感到困惑这个带有 OUTPUT 子句内容的临时表不知何故?我上面的语句非常好(感谢上帝)我最近添加了一个额外的 MERGE cond.for WHEN NOT MATCHED BY SOURCE 来解决传入数据中缺少的行。由于它缺少源数据,因此导致 UPDATE它实际上使用 UPDATE 语句退出 DIM 表中的行,不会将行添加到我用于最终 INSERT 的#TheTempTableName"。
    猜你喜欢
    • 1970-01-01
    • 1970-01-01
    • 1970-01-01
    • 2014-02-05
    • 1970-01-01
    • 1970-01-01
    • 1970-01-01
    • 1970-01-01
    • 1970-01-01
    相关资源
    最近更新 更多