为大数据选择 MySQL 上的查询优化答案

【问题标题】：Select Query Optimization on MySQL for Huge Data为大数据选择 MySQL 上的查询优化
【发布时间】：2013-08-04 15:50:33
【问题描述】：

## MySql Server 5.5，数据库引擎 MyIsam，表 fact_transaction 在 date_key、time_key、unit_cost_price、unit_retail_price（组合键）上建立索引，fact_stockout_sales 不包括 time_key。##

Query plan

查询

SELECT 
    t.Level, t.Name, t.KeyValue, 
    ROUND( (SUM(t.Gross)/SUM(t.Revenue))*100, 2 ) AS Value, 
    ROUND( (SUM(t.adjustedGross)/SUM(t.adjustedRevenue))*100, 2 ) AS adjustedValue,
    t.dataType AS dataType 
FROM 
   (SELECT "item" AS Level, ds.product_name AS Name, ds.product_id AS KeyValue, 
        SUM(ft.gross_profit) AS Gross, 
        SUM(ft.selling_amount) AS Revenue, 
        SUM(ft.adjusted_gross_profit) AS adjustedGross, 
        SUM(ft.adjusted_selling_amount) AS adjustedRevenue,
        "%" AS dataType 
    FROM fact_transaction AS ft 
    JOIN dim_sku AS ds ON ft.sku_key = ds.sku_key 
    WHERE ft.date_key BETWEEN 20080215 AND 20130107 
      AND ft.time_key BETWEEN 100 AND 235900 
      AND ft.unit_cost_price BETWEEN 0 AND 1333 
      AND ft.unit_retail_price BETWEEN 0 AND 16500 
      AND ft.store_key IN ("1", "2", "3", "4", "5", "6", "7", "8", "9", "10", "11", "12", "13", "14", "15", "16") 
      AND ds.product_id IN (1312009,1312007,... Huge List say 30000) 
      AND ds.category IN ("Male","Female","Unisex") 
      AND ft.day_of_week IN ("1","2","3","4","5","6","7") 
      AND ds.collection_name IN ("Base","SS12","AW12") 
    GROUP BY ds.product_id                          
    UNION 
    SELECT "item" AS Level, ds.product_name AS Name, ds.product_id AS KeyValue, 
        SUM(ft.gross_profit) AS Gross, 
        SUM(ft.selling_amount) AS Revenue, 
        SUM(ft.adjusted_gross_profit) AS adjustedGross, 
        SUM(ft.adjusted_selling_amount) AS adjustedRevenue, 
        "%" AS dataType 
    FROM fact_stockout_sales AS ft 
    JOIN dim_sku AS ds ON ft.sku_key = ds.sku_key 
    WHERE ft.date_key BETWEEN 20080215 AND 20130107 
      AND ft.unit_cost_price BETWEEN 0 AND 1333 
      AND ft.unit_retail_price BETWEEN 0 AND 16500 
      AND ft.store_key IN ("1", "2", "3", "4", "5", "6", "7", "8", "9", "10", "11", "12", "13", "14", "15", "16") 
      AND ds.product_id IN (1312009,1312007,.. Huge List say 30000) 
      AND ds.category IN ("Male","Female","Unisex") 
      AND ft.day_of_week IN ("1","2","3","4","5","6","7") 
      AND ds.collection_name IN ("Base","SS12","AW12") 
      GROUP BY ds.product_id) AS t 
GROUP BY t.KeyValue

【问题讨论】：

为什么使用 UNION 来组合两个 identic 查询？这一定很慢 - 相同的查询运行两次，然后 UNION 结合两个相同的结果集结束消除重复。
@kordirko：它们并不相同。他们从两个不同的表中进行选择，第二个从 WHERE 子句中省略了 ft.time_key。
对，它们不一样。但由于两个子查询都包含对同一个表 dim_sku 的连接，您可能会尝试使用“连接分解”来转换此查询以避免扫描 dim_sk 两次，此链接描述了此转换的工作原理：docs.oracle.com/cd/E16655_01/server.121/e15858/…
@Kordirko 感谢您提供的链接，我将对其进行调查。我的疑问是即使检测到组合索引它实际上并没有使用它。查看附加的查询计划。如何强制使用索引

标签： mysql select query-optimization union

【解决方案1】：

测量运行时间。
测量作为 UNION 运算符一部分的每个查询的运行时间。
索引 WHERE 子句中使用的每一列。
把选择性最高的列放在WHERE子句的前面，测试多列索引的效果。
减少无用的测试。（当然 ft.day_of_week 可以从 WHERE 子句中删除。）
重新考虑您的数据类型。星期几和存储键真的是字符串吗？
重新考虑一次选择五年数据的决定。
尝试将产品 ID 编号移动到临时表中，然后加入其中。

【讨论】：

1.Date,time,costPrice,retailPrice 是最主要的列，这就是为什么它会立即出现在 where 之后。 2.day_of_week, store_id,product_id 的数据类型只有整数。 3.Combined Indx 正在为主要列（数据、时间、成本价格、零售价格）进行 4.我粘贴的查询是除了 product_id 检查之外的确切查询，因为它要检查的列表很大。
Predominant 和 selective 在英语中不是同一个意思。您想在 WHERE 子句中使用的 selective 列上构建复合索引。我对确切查询的评论是我的错误——我认为括号不匹配。