优化 SQL 查询以删除游标

Question

我正在尝试编写一个查询，该查询将通过一个表并将帐户上的任何信用应用于最旧的余额。如果不使用游标，我想不出办法来做到这一点，而且我知道如果可能的话，应该不惜一切代价避免使用游标，所以我来这里寻求帮助。

select * into #balances from [IDAT_AR_BALANCES] where amount > 0
select * into #credits from [IDAT_AR_BALANCES] where amount < 0

create index ba_ID on #balances (CLIENT_ID)
create index cr_ID on #credits (CLIENT_ID)

declare credit_cursor cursor for
select [CLIENT_ID], amount, cvtGUID from #credits

open credit_cursor
declare @client_id varchar(11)
declare @credit money
declare @balance money
declare @cvtGuidBalance uniqueidentifier
declare @cvtGuidCredit uniqueidentifier
fetch next from credit_cursor into @client_id, @credit, @cvtGuidCredit
while @@fetch_status = 0
begin
      while(@credit < 0 and (select count(*) from #balances where @client_id = CLIENT_ID and amount <> 0) > 0)
      begin
            select top 1  @balance = amount, @cvtGuidBalance = cvtGuid from #balances where @client_id = CLIENT_ID and amount <> 0 order by AGING_DATE
            set @credit = @balance + @credit
            if(@credit > 0)
            begin
                  update #balances set amount = @credit where cvtGuid = @cvtGuidBalance
                  set @credit = 0
            end
            else
            begin
                  update #balances set amount = 0 where cvtGuid = @cvtGuidBalance
            end
      end
      update #credits set amount = @credit where cvtGuid = @cvtGuidCredit
      fetch next from credit_cursor into @client_id, @credit, @cvtGuidCredit
end

close credit_cursor
deallocate credit_cursor

delete #balances where AMOUNT = 0
delete #credits where AMOUNT = 0

truncate table [IDAT_AR_BALANCES]

insert [IDAT_AR_BALANCES] select * from #balances
insert [IDAT_AR_BALANCES] select * from #credits

drop table #balances
drop table #credits

在我的 10000 条记录和 1000 个客户端的测试用例中，运行需要 26 秒，通过添加 CLIENT_ID 上的两个索引，我能够将数字降低到 14 秒。但是这对于我的需要来说仍然太慢了，最终结果可能有多达 10000 个客户端和超过 4,000,000 条记录，因此运行时间很容易变成两位数分钟。

任何关于如何重组它以删除光标的建议将不胜感激。

示例（更新后表明您可以在运行后获得多个学分）：

before
cvtGuid      client_id      ammount     AGING_DATE
xxxxxx       1              20.00       1/1/2011
xxxxxx       1              30.00       1/2/2011
xxxxxx       1              -10.00      1/3/2011
xxxxxx       1              5.00        1/4/2011
xxxxxx       2              20.00       1/1/2011
xxxxxx       2              15.00       1/2/2011
xxxxxx       2              -40.00      1/3/2011
xxxxxx       2              5.00        1/4/2011
xxxxxx       3              10.00       1/1/2011
xxxxxx       3              -20.00      1/2/2011
xxxxxx       3              5.00        1/3/2011
xxxxxx       3              -8.00       1/4/2011

after
cvtGuid      client_id      ammount     AGING_DATE
xxxxxx       1              10.00       1/1/2011
xxxxxx       1              30.00       1/2/2011
xxxxxx       1              5.00        1/4/2011
xxxxxx       3              -5.00       1/2/2011
xxxxxx       3              -8.00       1/4/2011

所以它会将负信用应用于最旧的正余额（示例中的客户端 1），如果在完成后没有剩余的正余额，它会留下剩余的负余额（客户端 3），如果他们完全取消（90% 的时间都是真实数据的情况），它将完全删除记录（客户端 2）。

Answer 1

可以在递归 CTE 的帮助下解决这个问题。

基本思路是这样的：

1. 分别获取每个帐户的正负值的总和 (client_id)。

2. 根据amount 的符号和绝对值，遍历每个帐户并“捏掉”两个总数之一的金额（即永远不要“捏掉”相应总数超过其当前价值）。 amount 应该添加/减去相同的值。

3. 更新后删除amount变成0的行。

对于我的解决方案，我借用了 Lieven 的表变量定义（谢谢！），添加一列（cvtGuid，声明为 int 用于演示）和一行（原始的最后一行Lieven 的脚本中缺少的示例）。

/* preparing the demonstration data */
DECLARE @IDAT_AR_BALANCES TABLE (
  cvtGuid int IDENTITY,
  client_id INTEGER
  , amount FLOAT
  , date DATE
);
INSERT INTO @IDAT_AR_BALANCES
  SELECT 1, 20.00, '1/1/2011'
  UNION ALL SELECT 1, 30.00, '1/2/2011'
  UNION ALL SELECT 1, -10.00, '1/3/2011'
  UNION ALL SELECT 1, 5.00, '1/4/2011'
  UNION ALL SELECT 2, 20.00, '1/1/2011'
  UNION ALL SELECT 2, 15.00, '1/2/2011'
  UNION ALL SELECT 2, -40.00, '1/3/2011'
  UNION ALL SELECT 2, 5.00, '1/4/2011'
  UNION ALL SELECT 3, 10.00, '1/1/2011'
  UNION ALL SELECT 3, -20.00, '1/2/2011'
  UNION ALL SELECT 3, 5.00, '1/3/2011'
  UNION ALL SELECT 3, -8.00, '1/4/2011';

/* checking the original contents */
SELECT * FROM @IDAT_AR_BALANCES;

/* getting on with the job: */
WITH totals AS (
  SELECT
    /* 1) preparing the totals */
    client_id,
    total_pos = SUM(CASE WHEN amount > 0 THEN amount END),
    total_neg = SUM(CASE WHEN amount < 0 THEN amount END)
  FROM @IDAT_AR_BALANCES
  GROUP BY client_id
),
refined AS (
  /* 2) refining the original data with auxiliary columns:
     * rownum - row numbers (unique within accounts);
     * amount_to_discard_pos - the amount to discard `amount` completely if it's negative;
     * amount_to_discard_neg - the amount to discard `amount` completely if it's positive
  */
  SELECT
    *,
    rownum = ROW_NUMBER() OVER (PARTITION BY client_id ORDER BY date),
    amount_to_discard_pos = CAST(CASE WHEN amount < 0 THEN -amount ELSE 0 END AS float),
    amount_to_discard_neg = CAST(CASE WHEN amount > 0 THEN -amount ELSE 0 END AS float)
  FROM @IDAT_AR_BALANCES
),
prepared AS (
  /* 3) preparing the final table (using a recursive CTE) */
  SELECT
    cvtGuid = CAST(NULL AS int),
    client_id,
    amount = CAST(NULL AS float),
    date = CAST(NULL AS date),
    amount_update = CAST(NULL AS float),
    running_balance_pos = total_pos,
    running_balance_neg = total_neg,
    rownum = CAST(0 AS bigint)
  FROM totals
  UNION ALL
  SELECT
    n.cvtGuid,
    n.client_id,
    n.amount,
    n.date,
    amount_update = CAST(
      CASE
        WHEN n.amount_to_discard_pos < p.running_balance_pos
        THEN n.amount_to_discard_pos
        ELSE p.running_balance_pos
      END
      +
      CASE
        WHEN n.amount_to_discard_neg > p.running_balance_neg
        THEN n.amount_to_discard_neg
        ELSE p.running_balance_neg
      END
    AS float),
    running_balance_pos = CAST(p.running_balance_pos -
      CASE
        WHEN n.amount_to_discard_pos < p.running_balance_pos
        THEN n.amount_to_discard_pos
        ELSE p.running_balance_pos
      END
    AS float),
    running_balance_neg = CAST(p.running_balance_neg -
      CASE
        WHEN n.amount_to_discard_neg > p.running_balance_neg
        THEN n.amount_to_discard_neg
        ELSE p.running_balance_neg
      END
    AS float),
    n.rownum
  FROM refined n
    INNER JOIN prepared p ON n.client_id = p.client_id AND n.rownum = p.rownum + 1
)
/*                  -- some junk that I've forgotten to clean up,
SELECT *            -- which you might actually want to use
FROM prepared       -- to view the final prepared result set
WHERE rownum > 0    -- before actually running the update
ORDER BY client_id, rownum
*/
/* performing the update */
UPDATE t
SET amount = t.amount + u.amount_update
FROM @IDAT_AR_BALANCES t INNER JOIN prepared u ON t.cvtGuid = u.cvtGuid
OPTION (MAXRECURSION 0);

/* checking the contents after UPDATE */
SELECT * FROM @IDAT_AR_BALANCES;

/* deleting the eliminated amounts */
DELETE FROM @IDAT_AR_BALANCES WHERE amount = 0;

/* checking the contents after DELETE */
SELECT * FROM @IDAT_AR_BALANCES;

---

更新

正如 Lieven 正确建议的那样（再次感谢您！），您可以先删除帐户中 amount 加起来为 0 的所有行，然后然后更新其他行。这将提高整体性能，因为正如您所说，大多数数据的数量加起来为 0。

以下是 Lieven 删除“零账户”解决方案的一种变体：

DELETE FROM @IDAT_AR_BALANCES
WHERE client_id IN (
  SELECT client_id
  FROM @IDAT_AR_BALANCES
  GROUP BY client_id
  HAVING SUM(amount) = 0
)

但请记住，更新后的 DELETE 仍然需要，因为更新可能会将某些 amount 值重置为 0。如果我是你，我可能会考虑创建触发器 FOR更新将自动删除amount = 0 所在的行。这样的解决方案并不总是可以接受的，但有时很好。这取决于您还可以对您的数据做什么。这也可能取决于它是完全属于您的项目还是还有其他维护者（他们不喜欢“神奇地”和意外消失的行）。

Answer 2

我最近整理了一些非常相似的东西。我没有找到一个真正简单的解决方案，它最终需要几百行，但我可以提供几点。

您可以将您的积分放入带有每个客户序列号的表格中：

CREATE TABLE #CreditsInSequence
  (
  Client_ID INT NOT NULL,
  Sequence  INT NOT NULL,
  PRIMARY KEY (ClientID, Sequence),
  Date      DATE NOT NULL,
  Amount    DECIMAL NOT NULL
  )
INSERT INTO #CreditsInSequence (Client_ID, Sequence, Date, Amount)
  SELECT
    client_id, ROW_NUMBER (PARTITION BY client_id, ORDER BY date) AS Sequence, date, amount
  FROM
    #credits

如果一个客户只有一个信用，他们将在表格中有一行，Sequence = 1。如果另一个客户有三个信用，他们将有三行，序列号为 1、2 和 3。您现在可以遍历这个临时表，并且您只需要与任何单个客户端拥有的最多积分相等的迭代次数。

DECLARE @MaxSeq INT = (SELECT MAX(Sequence) FROM #Credits)
DECLARE @Seq    INT = 1
WHILE @Seq <= @MaxSeq
  BEGIN
  -- Do something with this set of credits
  SELECT
    Client_ID, Date, Amount
  FROM
    #CreditsInSequence
  WHERE
    Sequence = @Seq

  SET @Seq += 1  -- Don't forget to increment the loop!
  END

就像使用光标一样，这让您可以按顺序操作，在继续处理第二个信用之前完全处理每个客户的第一个信用。作为奖励，根据我的经验，这种“假装 FOR 循环”通常比光标快。

为了确定应用每个信用的正确余额，我将从以下内容开始：

SELECT
  B.client_id,
  MIN(B.date) AS Date,
  B.amount - COALESCE(AC.Amount, 0.00) AS MaxAmountCreditable
FROM
  #balances AS B
  LEFT JOIN #AllocatedCredits AS AC ON B.BalanceID = AC.BalanceID
WHERE
  B.amount + COALESCE(AC.Amount, 0.00) > 0.00
GROUP BY
  B.client_id

您需要扩展最后一个查询以从该日期获取实际余额 ID（cvtGuid，如果我没看错的话），在#AllocatedCredits 中记录这些分配，处理信用足以支付的情况关闭多个余额等。

祝你好运，如果您需要任何帮助，请随时返回 SO！

Answer 3

您必须验证它是否会更快，但这是通过（主要）基于集合的操作而不是基于光标的操作来完成的。

测试数据

DECLARE @IDAT_AR_BALANCES TABLE (
  client_id INTEGER
  , amount FLOAT
  , date DATE
) 

INSERT INTO @IDAT_AR_BALANCES
  SELECT 1, 20.00, '1/1/2011'
  UNION ALL SELECT 1, 30.00, '1/2/2011'
  UNION ALL SELECT 1, -10.00, '1/3/2011'
  UNION ALL SELECT 1, 5.00, '1/4/2011'
  UNION ALL SELECT 2, 20.00, '1/1/2011'
  UNION ALL SELECT 2, 15.00, '1/2/2011'
  UNION ALL SELECT 2, -40.00, '1/3/2011'
  UNION ALL SELECT 2, 5.00, '1/4/2011'
  UNION ALL SELECT 3, 10.00, '1/1/2011'
  UNION ALL SELECT 3, -20.00, '1/2/2011'
  UNION ALL SELECT 3, 5.00, '1/3/2011' 

删除所有加起来为 0（90% 的数据）

  DELETE FROM @IDAT_AR_BALANCES
  FROM @IDAT_AR_BALANCES b
       INNER JOIN (
         SELECT client_id
         FROM   @IDAT_AR_BALANCES
         GROUP BY 
                client_id
         HAVING SUM(amount) = 0
       ) bd ON bd.client_id = b.client_id

剩余记录

DECLARE @Oldest TABLE (
  client_id INTEGER PRIMARY KEY CLUSTERED
  , date DATE
)

DECLARE @Negative TABLE (
  client_id INTEGER PRIMARY KEY CLUSTERED
  , amount FLOAT
)  

WHILE EXISTS (  SELECT  b.client_id
                        , MIN(b.amount) 
                FROM    @IDAT_AR_BALANCES b
                        INNER JOIN (
                          SELECT  client_id
                          FROM    @IDAT_AR_BALANCES
                          GROUP BY
                                  client_id
                          HAVING  COUNT(*) > 1
                        ) r ON r.client_id = b.client_id                
                WHERE   b.amount < 0 
                GROUP BY 
                        b.client_id 
                HAVING COUNT(*) > 0
             )
BEGIN

  DELETE FROM @Oldest
  DELETE FROM @Negative

  INSERT INTO @Oldest
    SELECT  client_id
            , date = MIN(date)
    FROM    @IDAT_AR_BALANCES 
    WHERE   amount > 0
    GROUP BY
            client_id

  INSERT INTO @Negative
    SELECT  b.client_id
            , amount = SUM(amount)
    FROM    @IDAT_AR_BALANCES b
            LEFT OUTER JOIN @Oldest o ON o.client_id = b.client_id AND o.date = b.date
    WHERE   amount < 0
            AND o.client_id IS NULL
    GROUP BY
            b.client_id

  UPDATE  @IDAT_AR_BALANCES
  SET     b.amount = b.amount + n.amount
  FROM    @IDAT_AR_BALANCES b
          INNER JOIN @Oldest o ON o.client_id = b.client_id AND o.date = b.date
          INNER JOIN @Negative n ON n.client_id = b.client_id

  DELETE FROM @IDAT_AR_BALANCES
  FROM    @IDAT_AR_BALANCES b
          LEFT OUTER JOIN @Oldest o ON o.client_id = b.client_id AND o.date = b.date
          INNER JOIN (
            SELECT  client_id
            FROM    @IDAT_AR_BALANCES
            GROUP BY
                    client_id
            HAVING  COUNT(*) > 1
          ) r ON r.client_id = b.client_id
  WHERE   amount < 0
          AND o.client_id IS NULL

END  

DELETE  FROM @IDAT_AR_BALANCES
WHERE   amount = 0          

SELECT  *
FROM    @IDAT_AR_BALANCES

Answer 4

首先，正如您所说，您应该只与那些有余额的客户打交道。
其次，您可以使用 WHILE 循环模拟游标功能..

这里是对代码的修改。我把计算的内容放在了外面，因为它们不是问题……如果你想让我完成代码，请告诉我

--first, only deal with those clients with balances
select CLIENT_ID into #ToDoList 
from [IDAT_AR_BALANCES]
group by CLIENT_ID
having sum(amount)!=0

--next, get the temp debit and credit tables just for the clients you are working on
select * into #balances from [IDAT_AR_BALANCES] where amount > 0 and CLIENT_ID IN (SELECT CLIENT_ID FROM #ToDoList)
select * into #credits from [IDAT_AR_BALANCES] where amount < 0 and CLIENT_ID IN (SELECT CLIENT_ID FROM #ToDoList)

--fine
create index ba_ID on #balances (CLIENT_ID)
create index cr_ID on #credits (CLIENT_ID)

--simulate a cursor... but much less resource intensive

declare @client_id varchar(11)

-- now loop through each client and perform their aging
while exists (select * from #ToDoList)
begin
    select top 1 @client_id = CLIENT_ID from #ToDoList 

    --perform your debit to credit matching and account aging here, per client

    delete from #TodoList where Client_ID=@client_ID
end

--clean up.. drop temp tables, etc

Answer 5

最后一个想法...实际上，我确实为几年前开发的大型 Pest Control CRM 编写了这段代码...我发现解决此问题的最有效解决方案是....NET CLR 存储过程

虽然我通常不惜一切代价避免使用 CLR Procs。但有时它们的性能优于 SQL。在这种情况下，带有数学计算的过程（逐行）查询在 CLR proc 中会快得多。

就我而言，它比 SQL 快得多。

仅供参考