【问题标题】:SQL Server 2014 : slow stored procedure execution timeSQL Server 2014:存储过程执行时间慢
【发布时间】:2015-07-03 07:56:55
【问题描述】:

我的表结构如下:

AuditUserMethods:

+---------------+---------------+----------+
|  ColumnName   |   DataType    | Nullable |
+---------------+---------------+----------+
| Id            | INT           | NOT NULL |
| CreatedDate   | DATETIME      | NOT NULL |
| ApiMethodName | NVARCHAR(MAX) | NOT NULL |
| Request       | NVARCHAR(MAX) | NOT NULL |
| Result        | NVARCHAR(MAX) | NOT NULL |
| Method_Id     | INT           | NOT NULL |
| User_Id       | INT           | NULL     |
+---------------+---------------+----------+

AuditUserMethodErrorCodes:

+--------------------+----------+----------+
|     ColumnName     | DataType | Nullable |
+--------------------+----------+----------+
| Id                 | INT      | NOT NULL |
| AuditUserMethod_Id | INT      | NOT NULL |
| ErrorCode          | INT      | NOT NULL |
+--------------------+----------+----------+

ID 是两个表中的 PK。存在一对多的关系。一个AuditUserMethod 可以有多个AuditUserMethodErrorCodes。因此,FK AuditUserMethod_Id

AuditUserMethods 表中的AuditUserMethod_IdCreatedDate 都有两个非聚集索引。

该过程的目的是返回基于过滤器的分页结果集。 @PageSize 确定要返回多少行,@PageIndex 确定要返回哪个页面。所有其他变量都用于过滤。

返回三个结果集。

  1. 包含AuditUserMethods 详细信息
  2. 包含AuditUserMethodErrorCodes 详细信息
  3. 包含找到的总行数(即,如果页面大小为 1000 并且有 5000 行符合所有条件,则返回 5000)。

存储过程:

CREATE PROCEDURE [api].[Audit_V1_GetAuditDetails]
(
    @Users XML = NULL,
    @Methods XML = NULL,
    @ErrorCodes XML = NULL,
    @FromDate DATETIME = NULL,
    @ToDate DATETIME = NULL,
    @PageSize INT = 5,
    @PageIndex INT = 0
)
AS
BEGIN
    DECLARE @UserIds            TABLE   (Id INT)
    DECLARE @MethodNames        TABLE   (Name NVARCHAR(256))
    DECLARE @ErrorCodeIds       TABLE   (Id INT)

    DECLARE @FilterUsers        BIT = 0
    DECLARE @FilterMethods      BIT = 0
    DECLARE @FilterErrorCodes   BIT = 0

    INSERT @UserIds
       SELECT
           x.y.value('.', 'int')
       FROM 
           @Users.nodes('Ids/x/@i') AS x (y)

    INSERT @MethodNames
       SELECT
           x.y.value('.', 'NVARCHAR(256)')
       FROM 
           @Methods.nodes('ArrayOfString/string') AS x (y)

    INSERT @ErrorCodeIds
       SELECT
           x.y.value('.', 'int')
       FROM 
           @ErrorCodes.nodes('Ids/x/@i') AS x (y)

    IF EXISTS (SELECT TOP 1 0 FROM @UserIds)
       SET @FilterUsers = 1

    IF EXISTS (SELECT TOP 1 0 FROM @MethodNames)
       SET @FilterMethods = 1

    IF EXISTS (SELECT TOP 1 0 FROM @ErrorCodeIds)
       SET @FilterErrorCodes = 1

    DECLARE @StartRow INT = @PageIndex * @Pagesize

    DECLARE @PageDataResults TABLE (Id INT,
                                    CreatedDate DATETIME,
                                    ApiMethodName NVARCHAR(256), 
                                    Request NVARCHAR(MAX),
                                    Result NVARCHAR(MAX),
                                    MethodId INT,
                                    UserId INT,
                                    TotalRows INT);

    WITH PageData AS
    (
        SELECT
            id AS id
            , createddate AS createddate
            , apimethodname AS apimethodname
            , request AS request
            , result AS result
            , method_id AS method_id
            , user_id AS user_id
            , ROW_NUMBER() OVER (ORDER BY createddate DESC, id DESC) AS row_number
            , COUNT(*) OVER() as TotalRows
        FROM 
            dbo.AuditUserMethods AS aum
        WHERE 
           (@FromDate IS NULL OR 
            (@FromDate IS NOT NULL AND aum.createddate > @FromDate))
           AND (@ToDate IS NULL OR 
                (@ToDate IS NOT NULL AND aum.createddate < @ToDate))
           AND (@FilterUsers = 0 OR 
                (@FilterUsers = 1 AND aum.user_id IN (SELECT Id FROM @UserIds)))
           AND (@FilterMethods = 0 OR 
                (@FilterMethods = 1 AND aum.ApiMethodName IN (SELECT Name FROM @MethodNames)))
           AND (@FiltererRorCodes = 0 OR 
                    (@FiltererRorCodes = 1 
                     AND EXISTS (SELECT 1
                                 FROM AuditUserMethodErrorCodes e
                                 WHERE e.AuditUserMethod_Id = aum.Id
                                   AND e.ErrorCode IN (SELECT Id FROM @ErrorCodeIds)
                                )
                    )
               )
    )

    INSERT @PageDataResults
        SELECT TOP (@Pagesize)
            PageData.id AS id
            , PageData.createddate AS createddate
            , PageData.apimethodname AS apimethodname
            , PageData.request AS request
            , PageData.result AS result
            , PageData.method_id AS method_id
            , PageData.user_id AS user_id
            , PageData.TotalRows AS totalrows
         FROM 
             PageData
         WHERE 
             PageData.row_number > @StartRow
         ORDER BY 
             PageData.createddate DESC

    SELECT 
        Id, CreatedDate, ApiMethodName, Request, Result, MethodId, UserId
    FROM 
        @PageDataResults

    SELECT 
        aumec.AuditUserMethod_Id, aumec.ErrorCode
    FROM 
        @PageDataResults ps
    INNER JOIN 
        AuditUserMethodErrorCodes aumec ON ps.Id = aumec.AuditUserMethod_Id

    SELECT TOP 1 
        TotalRowsNumberOfReturnedAuditEntries 
    FROM @PageDataResults
END

AuditUserMethods 表包含 500,000 行,AuditUserMethodErrorCodes 包含 67843 行。

我正在使用以下参数执行该过程:

EXEC [api].[Audit_V1_GetAuditDetails]   @Users = N'<Ids><x i="1" /></Ids>'
                                        ,@Methods = NULL
                                        ,@ErrorCodes = N'<Ids />'
                                        ,@FromDate = '2015-02-15 07:18:59.613'
                                        ,@ToDate = '2015-07-02 08:18:59.613'
                                        ,@Pagesize = 5000
                                        ,@PageIndex = 0

存储过程只需 2 秒多的时间即可执行并返回 5000 行。我需要这个存储过程运行得更快,但我不知道如何改进它。

根据实际执行计划。相对于批次,占 99% 的是 CTE。在 CTE 中,Sort 占据了 95% 的成本:

【问题讨论】:

  • 第一次执行是否很快(服务器重新启动后或运行DBCC FREEPROCCACHE 后),然后在更改参数时立即变慢?或者,它只是慢?
  • 即使在释放 proc 缓存之后似乎也相当一致。
  • 我终于完成了我的回答stackoverflow.com/a/31202079/659190

标签: sql tsql stored-procedures sql-server-2014 sqlperformance


【解决方案1】:

我先声明几个表参数类型。

CREATE TYPE [api].[IdSet] AS TABLE
(
    [Id] INT NOT NULL
);

和,

CREATE TYPE [api].[StringSet] AS TABLE
(
    [Value] NVARCHAR(256) NOT NULL
);

然后我会更改存储过程的签名以使用它们。

注意我还将总计数作为输出参数返回,而不是作为单独的结果集。

CREATE PROCEDURE [api].[Audit_V2_GetAuditDetails]
(
    @userIds [api].[IdSet] READONLY,
    @methodNames [api].[StringSet] READONLY,
    @errorCodeIds [api].[IdSet] READONLY,
    @fromDate DATETIME = NULL,
    @toDate DATETIME = NULL,
    @pageSize INT = 5,
    @pageIndex INT = 0,
    @totalCount BIGINT OUTPUT
)

我知道您可能仍需要进行 XML 提取,但如果您在 SP 之外进行,它将有助于查询规划器。

现在,在 SP 中,我不会使用 @PageDataResults 我只会获取页面的 ID。我也不会使用 CTE,这在这种情况下没有帮助。

我会简化查询并运行一次以聚合总计数,然后如果大于 0,则再次运行相同的查询以仅返回 id 页面。查询的主体将被服务器内部缓存。

此外,Id' 使用 OFFSETFETCH 扩展至 ORDER BY 进行分页,

我在下面概述了许多逻辑简化,

CREATE PROCEDURE [api].[Audit_V2_GetAuditDetails]
    (
        @userIds [api].[IdSet] READONLY,
        @methodNames [api].[StringSet] READONLY,
        @errorCodeIds [api].[IdSet] READONLY,
        @fromDate DATETIME = NULL,
        @toDate DATETIME = NULL,
        @pageSize INT = 5,
        @pageIndex INT = 0,
        @totalCount BIGINT OUTPUT
    )
AS

DECLARE @offset INT = @pageSize * @pageIndex; 
DECLARE @filterUsers BIT = 0;
DECLARE @filterMethods BIT = 0;
DECLARE @filterErrorCodes BIT = 0;

IF EXISTS (SELECT 0 FROM @userIds)
    SET @filterUsers = 1;
IF EXISTS (SELECT 0 FROM @methodNames)
    SET @filterMethods = 1;
IF EXISTS (SELECT 0 FROM @errorCodeIds)
    SET @filterErrorCodes = 1;

SELECT
            @totalCount = COUNT_BIG(*)
     FROM
            [dbo].[AuditUserMethods] [aum]
         LEFT JOIN
            @userIds [U]
                ON [U].[Id] = [aum].[user_id]
         LEFT JOIN
            @methodName [M]
                ON [M].[Value] = [aum].[ApiMethodName]
    WHERE
            (
               @fromDate IS NULL
            OR
               [aum].[createddate] > @fromDate
            )
        AND
            (
               @toDate IS NULL
            OR
               [aum].[createddate] < @toDate
            )
        AND
            (
                @filterUsers = 0
            OR
                [U].[Id] IS NOT NULL
            (
        AND
            (
                @filterMethods = 0
            OR
                [M].[Value] IS NOT NULL
            (
        AND
            (
                @filterErrorCodes = 0
            OR 
                (
                    EXISTS(
                        SELECT
                                    1
                            FROM
                                    [dbo].[AuditUserMethodErrorCodes] [e]
                                JOIN
                                    @errorCodeIds [ec]
                                       ON [ec].[Id] = [e].[ErrorCode]
                            WHERE
                                    [e].[AuditUserMethod_Id] = [aum].[Id])
                );

DECLARE @pageIds [api].[IdSet];

IF @totalCount > 0
INSERT @pageIds
SELECT
            [aum].[id]
     FROM
            [dbo].[AuditUserMethods] [aum]
         LEFT JOIN
            @userIds [U]
                ON [U].[Id] = [aum].[user_id]
         LEFT JOIN
            @methodName [M]
                ON [M].[Value] = [aum].[ApiMethodName]
    WHERE
            (
               @fromDate IS NULL
            OR
               [aum].[createddate] > @fromDate
            )
        AND
            (
               @toDate IS NULL
            OR
               [aum].[createddate] < @toDate
            )
        AND
            (
                @filterUsers = 0
            OR
                [U].[Id] IS NOT NULL
            (
        AND
            (
                @filterMethods = 0
            OR
                [M].[Value] IS NOT NULL
            (
        AND
            (
                @filterErrorCodes = 0
            OR 
                (
                    EXISTS(
                        SELECT
                                    1
                            FROM
                                    [dbo].[AuditUserMethodErrorCodes] [e]
                                JOIN
                                    @errorCodeIds [ec]
                                       ON [ec].[Id] = [e].[ErrorCode]
                            WHERE
                                    [e].[AuditUserMethod_Id] = [aum].[Id])
                )
    ORDER BY
            [aum].[createddate] DESC,
            [aum].[id] DESC
        OFFSET @offset ROWS
        FETCH NEXT @pageSize ROWS ONLY;

SELECT 
            [aum].[Id],
            [aum].[CreatedDate],
            [aum].[ApiMethodName],
            [aum].[Request],
            [aum].[Result],
            [aum].[MethodId],
            [aum].[UserId]
    FROM
            [dbo].[AuditUserMethods] [aum]
    JOIN
            @pageIds [i]
                ON [i].[Id] = [aum].[id] 
ORDER BY
            [aum].[createddate] DESC,
            [aum].[id] DESC;

SELECT 
            [aumec].[AuditUserMethod_Id],
            [aumec].[ErrorCode]
    FROM
            [dbo].[AuditUserMethodErrorCodes] [aumec]
        JOIN
            @pageIds [i]
                ON [i].[Id] = [aumec].[AuditUserMethod_Id];

/* The total count is an output parameter */
RETURN 0;

如果这还不足以改善情况,您需要查看查询计划并考虑哪些索引是最佳的。

警告所有的代码都是即兴写的,所以,虽然想法是正确的,但语法可能并不完美。

【讨论】:

    【解决方案2】:
    (@FromDate IS NULL OR 
                (@FromDate IS NOT NULL AND aum.createddate > @FromDate))
    

    一样
    (@FromDate IS NULL OR  aum.createddate > @FromDate)
    

    试试这样的

    CREATE PROCEDURE [api].[Audit_V1_GetAuditDetails]
    (
        @Users XML = NULL,
        @Methods XML = NULL,
        @ErrorCodes XML = NULL,
        @FromDate DATETIME = NULL,
        @ToDate DATETIME = NULL,
        @PageSize INT = 5,
        @PageIndex INT = 0
    )
    AS
    BEGIN
        DECLARE @UserIds            TABLE   (Id INT)
        DECLARE @MethodNames        TABLE   (Name NVARCHAR(256))
        DECLARE @ErrorCodeIds       TABLE   (Id INT)
    
        INSERT @UserIds
           SELECT
               x.y.value('.', 'int')
           FROM 
               @Users.nodes('Ids/x/@i') AS x (y)
    
        INSERT @MethodNames
           SELECT
               x.y.value('.', 'NVARCHAR(256)')
           FROM 
               @Methods.nodes('ArrayOfString/string') AS x (y)
    
        INSERT @ErrorCodeIds
           SELECT
               x.y.value('.', 'int')
           FROM 
               @ErrorCodes.nodes('Ids/x/@i') AS x (y)
    
        IF NOT EXISTS (SELECT TOP 1 0 FROM @UserIds)
           INSERT INTO @UserIds values (-1)
    
        IF NOT EXISTS (SELECT TOP 1 0 FROM @MethodNames)
           INSERT INTO @MethodNames values ('empty')
    
        IF NOT EXISTS (SELECT TOP 1 0 FROM @ErrorCodeIds)
           INSERT INTO @ErrorCodeIds values (-1)
    
        IF @FromDate is null 
           @FromDate = '1/1/1900'
    
        IF @ToDate is null 
           @ToDate = '1/1/2079'
    
        DECLARE @StartRow INT = @PageIndex * @Pagesize
    
        DECLARE @PageDataResults TABLE (Id INT,
                                        CreatedDate DATETIME,
                                        ApiMethodName NVARCHAR(256), 
                                        Request NVARCHAR(MAX),
                                        Result NVARCHAR(MAX),
                                        MethodId INT,
                                        UserId INT,
                                        TotalRows INT);
    
        WITH PageData AS
        (
            SELECT
                id AS id
                , createddate AS createddate
                , apimethodname AS apimethodname
                , request AS request
                , result AS result
                , method_id AS method_id
                , user_id AS user_id
                , ROW_NUMBER() OVER (ORDER BY createddate DESC, id DESC) AS row_number
                , COUNT(*) OVER() as TotalRows
            FROM 
                dbo.AuditUserMethods AS aum
            JOIN @UserIds
              ON (aum.user_id = @UserIds.ID OR @UserIds.ID = -1)
             AND aum.createddate > @FromDate
             AND aum.createddate < @ToDate
            JOIN @MethodNames 
              ON aum.ApiMethodName = @MethodNames.Name 
              OR @MethodNames.Name = 'empty'
            JOIN AuditUserMethodErrorCodes e
              on e.AuditUserMethod_Id = aum.Id 
            JOIN @ErrorCodeIds 
              ON e.ErrorCode = @ErrorCodeIds.ID 
              OR @ErrorCodeIds.ID = -1
        )
    

    【讨论】:

    • 我喜欢这个主意。它在 WHERE 子句中看起来更清晰。至于 OR 语句是一样的。我不确定 SQL 是否支持短路操作。
    • 哪里有问题,你可以得到一个大的加入然后过滤。这个想法是帮助查询优化器及早过滤。这并不是真正的短路 - 与 null 的比较无论如何都会返回 false。
    【解决方案3】:

    您可以从给您的 CTE 某种索引开始,这可以通过以下方式完成 - 请参阅 /**/ 以了解“更改的行”:

    WITH PageData AS
    (
        SELECT
    /**/    TOP 100 PERCENT                                                                 
            id                                              AS id
            ,createddate                                    AS createddate
            ,apimethodname                                  AS apimethodname
            ,request                                        AS request
            ,result                                         AS result
            ,method_id                                      AS method_id
            ,user_id                                        AS user_id
            ,ROW_NUMBER() OVER (ORDER BY createddate DESC, id DESC) AS row_number
            ,COUNT(*) OVER() as TotalRows
        FROM dbo.AuditUserMethods AS aum
        WHERE (@FromDate IS NULL OR (@FromDate IS NOT NULL AND aum.createddate > @FromDate))
        AND (@ToDate IS NULL OR (@ToDate IS NOT NULL AND aum.createddate < @ToDate))
        AND (@FilterUsers = 0 OR (@FilterUsers = 1 AND aum.user_id IN (SELECT Id FROM @UserIds)))
        AND (@FilterMethods = 0 OR (@FilterMethods = 1 AND aum.ApiMethodName IN (SELECT Name FROM @MethodNames)))
        AND 
            (
                @FiltererRorCodes = 0 OR 
                    (
                        @FiltererRorCodes = 1 AND EXISTS 
                            (
                                SELECT 1
                                FROM AuditUserMethodErrorCodes e
                                WHERE e.AuditUserMethod_Id = aum.Id
                                AND e.ErrorCode IN (SELECT Id FROM @ErrorCodeIds)
                            )
                    )
            )
    /**/ORDER BY 
    /**/    PageData.createddate 
    /**/    ,PageData.row_number
    )
    

    我还将尝试在 createddate 和 row_number 之间的 CTE 上更改 order 的“顺序”,然后首先是 row_number,然后是 createddate。

    然后,您将 CTE 交给下一个流程,它已经按照预期的顺序进行。它可能会加快速度。 ORDER BY 需要 TOP 100 PERCENT。

    【讨论】:

    • 像这样的技巧被称为“中间物化 CTE”,需要额外的排序。我个人更喜欢(本地)临时表中的“物化”。额外的好处是这些提供统计信息,在某些情况下,您的查询可以从添加索引(或多个)中受益。
    • @TT 我有时会走表变量路线,尤其是当 SQL 给您带来延迟加载性能混乱时,表变量或临时表在您期望的时候完成。在这种情况下,我认为只需要一个“索引”,因此 CTE 可能运行良好并且需要很少的返工。感谢您的评论。
    • ` TOP 100 PERCENT` 无论如何都会被优化掉并且没有任何效果。
    • @MartinSmith 我已经广泛使用了这个“技巧”,如果肯定可以按预期工作。 CTE 的 order by 可加快查询速度,但需要 TOP 100 PERCENT,因为您将获得“Msg 1033, Level 15, State 1, Line 10 ORDER BY 子句在视图、内联函数、派生表中无效,子查询和公用表表达式,除非还指定了 TOP 或 FOR XML。"
    猜你喜欢
    • 1970-01-01
    • 2015-06-14
    • 2012-04-27
    • 1970-01-01
    • 2018-09-01
    • 2011-11-07
    • 2014-05-10
    • 1970-01-01
    • 2015-02-26
    相关资源
    最近更新 更多