慢 UDF 的 SQL 替代方案答案

【问题标题】：SQL Alternatives to Slow UDF慢 UDF 的 SQL 替代方案
【发布时间】：2018-03-19 11:37:15
【问题描述】：

带有两个 UDT 参数的查询需要 0.3 秒，但是当封装在一个内联表值函数中时，它需要 3.5+ 秒。

我已阅读 (Why is a UDF so much slower than a subquery?)，但正在为如何修复/重写而苦苦挣扎。

根据下面@JasonALong 的反馈，

在 0.3 秒内完成的 SELECT 语句的执行计划：https://www.brentozar.com/pastetheplan/?id=HJnrqC53Z（请注意此页面上提供了 SQL）。

下面粘贴的 3.5 秒内完成的函数代码和此链接上的执行计划：https://www.brentozar.com/pastetheplan/?id=BJZbqR93b

SELECT
SelectedContracts.MeasurableID,
SelectedContracts.EntityID,

EntityName,
EntityAbbrev,
EntityLogoURL,
EntityHex1,
EntityHex2,
EntitySportID,

MeasurableName,
MeasurableOrganizationID,
YearFilter,
SeasonFilter,
CategoryFilter,
ResultFilter,
Logo4Result,
MeasurableSportID,
MouseoverFooter,
ContractRank4Org,
ContractEndUTC,

HighContractPrice4Period,
HighTradeID,
HighTradeUTC,
HighTradeNumberOfContracts,
HighTradeCurrency,

LowContractPrice4Period,
LowTradeID,
LowTradeUTC,
LowTradeNumberOfContracts,
LowTradeCurrency,

LastTradePrice,
LastTradeID,
LastTradeUTC,
LastTradeNumberOfContracts,
LastTradeCurrency,

SecondLastTradePrice,
SecondLastTradeID,
SecondLastTradeUTC,
SecondLastTradeNumberOfContracts,
SecondLastTradeCurrency,

ContractPrice4ChangeCalc,
ContractID4ChangeCalc,
ContractUTC4ChangeCalc,
ContractsNumberTraded4ChangeCalc,
ContractCurrency4ChangeCalc,

HighestBidID,
HighestBidMemberID,
HighestBidPrice,
HighestBidAvailableContracts,
HighestBidCurrency,

LowestAskID,
LowestAskMemberID,
LowestAskPrice,
LowestAskAvailableContracts,
LowestAskCurrency


FROM
(
    SELECT
        dbo.Contracts.MeasurableID,
        dbo.Contracts.EntityID
    FROM
        dbo.Contracts
    WHERE
        dbo.Contracts.MeasurableID IN ((2030),(2017),(2018),(2019),(2020),( 2028),(2024),(2027),(2029),(2022),( 4018),(4019),(4020),(4021))
    GROUP BY
        dbo.Contracts.MeasurableID,
        dbo.Contracts.EntityID
) SelectedContracts


INNER JOIN 
(
    SELECT
        dbo.Entities.ID,
        --dbo.Entities.OrganizationID, -- Get OrganizationID from Measurable since some Entities (European soccer teams) have multiple Orgs
        dbo.Entities.EntityName,
        dbo.Entities.EntityAbbrev,
        dbo.Entities.logoURL AS EntityLogoURL,
        dbo.Entities.Hex1 AS EntityHex1,
        dbo.Entities.Hex2 AS EntityHex2,
        dbo.Entities.SportID AS EntitySportID
    FROM
        dbo.Entities
) SelectedEntities ON SelectedContracts.EntityID = SelectedEntities.ID


INNER JOIN 
(
    SELECT
        dbo.Measurables.ID AS MeasurableID,
        dbo.Measurables.Name AS MeasurableName,
        dbo.Measurables.OrganizationID AS MeasurableOrganizationID,
        dbo.Measurables.[Year] AS YearFilter,
        dbo.Measurables.Season AS SeasonFilter,
        dbo.Measurables.Category AS CategoryFilter,
        dbo.Measurables.Result AS ResultFilter,
        dbo.Measurables.Logo4Result,
        dbo.Measurables.SportID AS MeasurableSportID,
        dbo.Measurables.MouseoverFooter,
        dbo.Measurables.ContractRank4Org,
        dbo.Measurables.EndUTC AS ContractEndUTC
    FROM
        dbo.Measurables
) MEASURABLES_table ON SelectedContracts.MeasurableID = MEASURABLES_table.MeasurableID


LEFT JOIN 
(
    SELECT
        MeasurableID,
        EntityID,
        ContractPrice AS HighContractPrice4Period,
        ID AS HighTradeID,
        UTCMatched AS HighTradeUTC,
        NumberOfContracts AS HighTradeNumberOfContracts,
        CurrencyCode AS HighTradeCurrency
    FROM
                (
                    SELECT
                        *, ROW_NUMBER () OVER (
                            PARTITION BY MeasurableID,
                            EntityID
                        ORDER BY
                            ContractPrice DESC,
                            ID DESC
                        ) RowNumber -- ID DESC means most recent trade of ties
                    FROM
                        Contracts
                    WHERE
                        MeasurableID IN ((2030),(2017),(2018),(2019),(2020),( 2028),(2024),(2027),(2029),(2022),( 4018),(4019),(4020),(4021))
                        AND dbo.Contracts.UTCmatched < DATEADD(DAY, -30, SYSDATETIME())
                        AND (           CurrencyCode IN (('GBP'), ('CAD'), ('INR'), ('BRL'), ('MXN'), ('CHF'), ('RUB')) 
                                )   
                ) AS InnerSelect4HighTrade

    WHERE   
        InnerSelect4HighTrade.RowNumber = 1

) HighTrades ON SelectedContracts.MeasurableID = HighTrades.MeasurableID AND SelectedContracts.EntityID = HighTrades.EntityID


LEFT JOIN 
(
    SELECT
        MeasurableID,
        EntityID,
        ContractPrice AS LowContractPrice4Period,
        ID AS LowTradeID,
        UTCMatched AS LowTradeUTC,
        NumberOfContracts AS LowTradeNumberOfContracts,
        CurrencyCode AS LowTradeCurrency
    FROM
        (
            SELECT
                    *, ROW_NUMBER () OVER (
                    PARTITION BY MeasurableID,
                    EntityID
                ORDER BY
                    ContractPrice ASC,
                    ID DESC
                ) RowNumber -- ID DESC means most recent trade of ties
            FROM
                Contracts
            WHERE
                MeasurableID IN ((2030),(2017),(2018),(2019),(2020),( 2028),(2024),(2027),(2029),(2022),( 4018),(4019),(4020),(4021))
                AND dbo.Contracts.UTCmatched < DATEADD(DAY, -30, SYSDATETIME())
                AND (           CurrencyCode IN (('GBP'), ('CAD'), ('INR'), ('BRL'), ('MXN'), ('CHF'), ('RUB')) 
                        )           
        ) AS InnerSelect4LowTrade

    WHERE       InnerSelect4LowTrade.RowNumber = 1

) LowTrades ON SelectedContracts.MeasurableID = LowTrades.MeasurableID AND SelectedContracts.EntityID = LowTrades.EntityID


LEFT JOIN 
(
    SELECT
        MeasurableID,
        EntityID,
        ContractPrice AS LastTradePrice,
        ID AS LastTradeID,
        UTCMatched AS LastTradeUTC,
        NumberOfContracts AS LastTradeNumberOfContracts,
        CurrencyCode AS LastTradeCurrency
    FROM
        (
            SELECT
                *, ROW_NUMBER () OVER (
                    PARTITION BY MeasurableID,
                    EntityID
                ORDER BY
                    ID DESC
                ) RowNumber -- ID DESC means most recent trade of ties
            FROM
                Contracts
            WHERE
                MeasurableID IN ((2030),(2017),(2018),(2019),(2020),( 2028),(2024),(2027),(2029),(2022),( 4018),(4019),(4020),(4021))
                AND (           CurrencyCode IN (('GBP'), ('CAD'), ('INR'), ('BRL'), ('MXN'), ('CHF'), ('RUB')) 
                        )   
        ) AS InnerSelect4LastTrade

    WHERE   InnerSelect4LastTrade.RowNumber = 1

) LastTrades ON SelectedContracts.MeasurableID = LastTrades.MeasurableID AND SelectedContracts.EntityID = LastTrades.EntityID


LEFT JOIN 
(
    SELECT
        MeasurableID,
        EntityID,
        ContractPrice AS SecondLastTradePrice,
        ID AS SecondLastTradeID,
        UTCMatched AS SecondLastTradeUTC,
        NumberOfContracts AS SecondLastTradeNumberOfContracts,
        CurrencyCode AS SecondLastTradeCurrency
    FROM
        (
            SELECT
                *, ROW_NUMBER () OVER (
                    PARTITION BY MeasurableID,
                    EntityID
                ORDER BY
                    ID DESC
                ) RowNumber -- ID DESC means most recent trade of ties
            FROM
                Contracts
            WHERE
                MeasurableID IN ((2030),(2017),(2018),(2019),(2020),( 2028),(2024),(2027),(2029),(2022),( 4018),(4019),(4020),(4021))
                AND (           CurrencyCode IN (('GBP'), ('CAD'), ('INR'), ('BRL'), ('MXN'), ('CHF'), ('RUB')) 
                        )   
--need time filter???
        ) AS InnerSelect4SecondToLastTrade

    WHERE InnerSelect4SecondToLastTrade.RowNumber = 2

) SecondToLastTrade ON SelectedContracts.MeasurableID = SecondToLastTrade.MeasurableID AND SelectedContracts.EntityID = SecondToLastTrade.EntityID


LEFT JOIN 
(
    SELECT
        MeasurableID,
        EntityID,
        ContractPrice AS ContractPrice4ChangeCalc,
        ID AS ContractID4ChangeCalc,
        UTCMatched AS ContractUTC4ChangeCalc,
        NumberOfContracts AS ContractsNumberTraded4ChangeCalc,
        CurrencyCode AS ContractCurrency4ChangeCalc
    FROM
        (
            SELECT
                *, ROW_NUMBER () OVER (
                    PARTITION BY MeasurableID,
                    EntityID
                ORDER BY
                    ID DESC  -- ID DESC equals the most recent trade if ties
                ) RowNumber 
            FROM
                Contracts
            WHERE
                MeasurableID IN ((2030),(2017),(2018),(2019),(2020),( 2028),(2024),(2027),(2029),(2022),( 4018),(4019),(4020),(4021))
                AND (           CurrencyCode IN (('GBP'), ('CAD'), ('INR'), ('BRL'), ('MXN'), ('CHF'), ('RUB')) 
                        )   
            AND dbo.Contracts.UTCmatched < DATEADD(Day ,-30, SYSDATETIME())
        ) AS InnerSelect4ChangeCalcPerPeriod

    WHERE   InnerSelect4ChangeCalcPerPeriod.RowNumber = 1

) Trade4ChangeCalcPerPeriod ON SelectedContracts.MeasurableID = Trade4ChangeCalcPerPeriod.MeasurableID AND SelectedContracts.EntityID = Trade4ChangeCalcPerPeriod.EntityID


LEFT JOIN 
(
    SELECT
        MeasurableID,
        EntityID,
        ID AS HighestBidID,
        MemberID AS HighestBidMemberID,
        BidPrice AS HighestBidPrice,
        AvailableContracts AS HighestBidAvailableContracts,
        CurrencyCode AS HighestBidCurrency
    FROM
        (
            SELECT
                *, ROW_NUMBER () OVER (
                    PARTITION BY MeasurableID,
                    EntityID
                ORDER BY
                    BidPrice DESC,
                    ID DESC
                ) RowNumber
            FROM
                dbo.Interest2Buy
            WHERE
                MeasurableID IN ((2030),(2017),(2018),(2019),(2020),( 2028),(2024),(2027),(2029),(2022),( 4018),(4019),(4020),(4021))
            AND AvailableContracts > 0
                AND (           CurrencyCode IN (('GBP'), ('CAD'), ('INR'), ('BRL'), ('MXN'), ('CHF'), ('RUB')) 
                        )   
        ) AS InnerSelect4HighestBid

    WHERE   InnerSelect4HighestBid.RowNumber = 1

) HighestBids ON SelectedContracts.MeasurableID = HighestBids.MeasurableID AND SelectedContracts.EntityID = HighestBids.EntityID


LEFT JOIN 
(
    SELECT
        MeasurableID,
        EntityID,
        ID AS LowestAskID,
        MemberID AS LowestAskMemberID,
        AskPrice AS LowestAskPrice,
        AvailableContracts AS LowestAskAvailableContracts,
        CurrencyCode AS LowestAskCurrency
    FROM
        (
            SELECT
                *, ROW_NUMBER () OVER (
                    PARTITION BY MeasurableID,
                    EntityID
                ORDER BY
                    AskPrice ASC,
                    ID DESC
                ) RowNumber
            FROM
                dbo.Interest2Sell
            WHERE
                MeasurableID IN ((2030),(2017),(2018),(2019),(2020),( 2028),(2024),(2027),(2029),(2022),( 4018),(4019),(4020),(4021))
                AND AvailableContracts > 0
                AND (           CurrencyCode IN (('GBP'), ('CAD'), ('INR'), ('BRL'), ('MXN'), ('CHF'), ('RUB')) 
                        )   
        ) AS InnerSelect4BestAsk

    WHERE   InnerSelect4BestAsk.RowNumber = 1

) BestAsks ON SelectedContracts.MeasurableID = BestAsks.MeasurableID AND SelectedContracts.EntityID = BestAsks.EntityID

【问题讨论】：

这太模糊了，您的 UDF 可以是标量函数或表值，它们可以是单语句或多语句，您可以将它们用作相关子查询或加入它们。名单还在继续。您需要给出一个与您的特定情况相关的实际示例，也许是在阅读本文之后？ stackoverflow.com/help/mcve （获取代码的两个版本的执行计划以查看它们的不同之处也不是一个坏主意，这可能会帮助您找出它们不同的原因）跨度>
对于跟随 fyi 的任何读者，我尝试添加 Option(Recompile) stackoverflow.com/questions/20864934/… 但这并没有什么不同。还尝试在 .Net（网络服务器）中创建 SQL 查询并直接运行，但事实证明这比使用函数或存储过程还要慢。
如果您使用 UDT 运行它，但未包装在函数中，会发生什么情况？这将帮助您隔离导致性能问题的更改（添加 UDT 或包装在函数中）。我怀疑这是问题所在的 UDT。如果是这样，请尝试重写查询以加入游戏，而不是使用 IN ()，或者将索引和统计信息应用于 UDT。

标签： sql-server stored-procedures user-defined-functions user-defined-types

【解决方案1】：

正如您在问题中提到的，标量函数和多语句表值函数 (mTVF) 都是优化器的“黑匣子”......

所以，我的问题是，“为什么这么糟糕？”。答案是，为了制定一个尽可能高效执行的好计划，它需要了解有关特定要求的某些细节以及它将从中提取数据的表的信息（这就是为什么过时的静态数据可以严重影响性能）。所以...当您使用标量函数或 mTVF 时，优化器无法像使用内联代码那样评估所有需求。它的反应是简单地假设该函数只会执行一次并根据该假设制定计划。

由于假设是错误的，因此会生成错误的计划，最终会导致糟糕的表现。

解决方案是重写有问题的函数...关键是#1，确保将它们重写为“内联表值函数”（iTVF）。这些是优化器将看到的唯一函数，就好像它们的代码直接输入到外部查询中一样（因此称为“内联”）。如果您不熟悉 iTVF，它们有 2 个要求... 1 它们必须是表函数（无论出于何种原因，MS 仍然没有可用的标量版本）... 2 这是biggie... 函数体必须是单个语句。

如果你不需要表值函数，你需要一个标量函数怎么办？好吧，没有什么说多值函数不能返回单个（标量）值......这就是为什么那些知道将所有函数代码作为 iTVF 的情况的原因。

好消息是网络上不乏有关创建“内联标量函数”的信息，使用编码的表函数在网络上返回标量值。

希望对您有所帮助...

【讨论】：

@JasonALong 谢谢，你的信息让我指向更好的研究。但是，与您评论的底部有关，我的问题是我想返回一个表而不是一个标量。（另外我认为不可能将没有 JOINS 的数据返回给子查询......）
很难说没有任何细节......但是......只是这样就不会混淆......当我说“作为一个单一的陈述”时，不应将其解释为“简单statement"...CTE、派生表和子查询都是完全可以接受的。它只是意味着你不能做一些事情，比如使用 IF 块添加控制流或声明和设置内部变量。这是一种简单的思考方式... SQL 语句应该以分号结尾... 如果您能够在正文中添加超过 1 个分号，SQL Server 将认为它是多值的。
如果要验证 SQL Server 是否认为特定函数“内联”，只需查询 sys.objects 表。 SELECT * FROM sys.objects o WHERE o.name = N'tfn_SomeFunction'; ...或...您可以一次查看所有 uds... SELECT * FROM sys.objects o WHERE o.type IN ('FN', 'FS', 'IF', 'TF');跨度>
这是一件好事。这意味着它不是一个黑匣子。然而，这并不意味着它对查询的其余部分有任何好处。这是您开始深入研究执行盘并找出正在发生的事情的时候。如果您不习惯解读执行计划，那么任何 SQL 论坛上总会有人喜欢深入研究执行计划。另外...... BrentOzar.com 和 SentryOne 都允许您上传您的计划，然后人们会为您审核。只要确保你捕捉到一个“实际计划”。如果您发布预估计划，系统仍会要求您提供实际计划。
优化器预计有 480 万行但只返回 64 行...至少有一个表是堆（无聚集索引），因为我看到 RID 查找...2 键查找 22表扫描，9 次排序操作...和一个哈希连接来结束它...我一直想说，“第一个让我震惊的”...重复迭代，所以我在考虑递归或交叉连接。您有机会将功能代码添加到问题中吗？

【解决方案2】：

使用联接而不是“IN”子句有很大帮助。（虽然我也将表 var 更改为临时表，这也有很大帮助。）

【讨论】：