SQL Server优化器特性-位图过滤(Bitmap)

一直以来,由于SQL Server中没有位图索引使得面对一些场景,从业人员在索引选择上受限,饱受诟病.其实熟悉SQL Server的朋友应该知道,SQL Server虽然没有位图索引,但在特定环境下还是会采用位图(Bitmap)过滤的,这次就为大家介绍下SQL Server的位图过滤.

概念:关于位图索引的概念我就不做过多介绍了,感兴趣的朋友可以看下wikipedia

http://en.wikipedia.org/wiki/Bitmap_index

优势:在重复率高,数据很少被更新的场景中(如一年之内的年龄,汽车车型等)过滤高效.

SQL Server的位图过滤采用的布隆过滤(bloom filter)方式,这里我简单说下布隆过滤的实现方式.

实现方式:通过构建一个长度X的位数组(bit array)(所有位为0),将要匹配的集合通过哈希函数映射到位数组中的相应点中(相应位为1),当判断一个值是否存在时找bit array中对应位是否为1就可以了.这个过程由SQL Server内部自己完成.

如图1-1所示,我将需要匹配的集合{神仙?,妖怪?,谢谢!}映射到bit array中,当有一条新记录{悟空..}我判断他是否在我的集合中,只需判断相应的位是否是1就可以了,图中可以看出{悟空..}并不是所有位都为1,所以悟空并不在我的集合中.

SQL Server优化器特性-位图过滤(Bitmap)

图1-1

具体到SQL Server中是如何实现的呢?我们还是通过一个实例来看.

测试环境脚本

USE AdventureWorks
GO

SELECT
    p.ProductID + (a.number * 1000) AS ProductID,
    p.Name + CONVERT(VARCHAR, (a.number * 1000)) AS Name,
    p.ProductNumber + '-' + CONVERT(VARCHAR, (a.number * 1000)) AS ProductNumber,
    p.MakeFlag,
    p.FinishedGoodsFlag,
    p.Color,
    p.SafetyStockLevel,
    p.ReorderPoint,
    p.StandardCost,
    p.ListPrice,
    p.Size,
    p.SizeUnitMeasureCode,
    p.WeightUnitMeasureCode,
    p.Weight,
    p.DaysToManufacture,
    p.ProductLine,
    p.Class,
    p.Style,
    p.ProductSubcategoryID,
    p.ProductModelID,
    p.SellStartDate,
    p.SellEndDate,
    p.DiscontinuedDate
INTO T1
FROM Production.Product AS p
CROSS JOIN master..spt_values AS a
WHERE
    a.type = 'p'
    AND a.number BETWEEN 1 AND 50
GO

SELECT 
    ROW_NUMBER() OVER 
    (
        ORDER BY 
            x.TransactionDate,
            (SELECT NEWID())
    ) AS TransactionID,
    p1.ProductID,
    x.TransactionDate,
    x.Quantity,
    CONVERT(MONEY, p1.ListPrice * x.Quantity * RAND(CHECKSUM(NEWID())) * 2) AS ActualCost
INTO T2
FROM
(
    SELECT
        p.ProductID, 
        p.ListPrice,
        CASE
            WHEN p.productid % 26 = 0 THEN 26
            WHEN p.productid % 25 = 0 THEN 25
            WHEN p.productid % 24 = 0 THEN 24
            WHEN p.productid % 23 = 0 THEN 23
            WHEN p.productid % 22 = 0 THEN 22
            WHEN p.productid % 21 = 0 THEN 21
            WHEN p.productid % 20 = 0 THEN 20
            WHEN p.productid % 19 = 0 THEN 19
            WHEN p.productid % 18 = 0 THEN 18
            WHEN p.productid % 17 = 0 THEN 17
            WHEN p.productid % 16 = 0 THEN 16
            WHEN p.productid % 15 = 0 THEN 15
            WHEN p.productid % 14 = 0 THEN 14
            WHEN p.productid % 13 = 0 THEN 13
            WHEN p.productid % 12 = 0 THEN 12
            WHEN p.productid % 11 = 0 THEN 11
            WHEN p.productid % 10 = 0 THEN 10
            WHEN p.productid % 9 = 0 THEN 9
            WHEN p.productid % 8 = 0 THEN 8
            WHEN p.productid % 7 = 0 THEN 7
            WHEN p.productid % 6 = 0 THEN 6
            WHEN p.productid % 5 = 0 THEN 5
            WHEN p.productid % 4 = 0 THEN 4
            WHEN p.productid % 3 = 0 THEN 3
            WHEN p.productid % 2 = 0 THEN 2
            ELSE 1 
        END AS ProductGroup
    FROM bigproduct p
) AS p1
CROSS APPLY
(
    SELECT
        transactionDate,
        CONVERT(INT, (RAND(CHECKSUM(NEWID())) * 100) + 1) AS Quantity
    FROM
    (
        SELECT 
            DATEADD(dd, number, '20050101') AS transactionDate,
            NTILE(p1.ProductGroup) OVER 
            (
                ORDER BY number
            ) AS groupRange
        FROM master..spt_values
        WHERE 
            type = 'p'
    ) AS z
    WHERE
        z.groupRange % 2 = 1
) AS x

View Code