使用 UNION 优化 MySQL 查询答案

【问题标题】：Optimize a MySQL query with UNIONs使用 UNION 优化 MySQL 查询
【发布时间】：2017-01-24 01:36:04
【问题描述】：

我需要运行一个包含多个 UNION 的长 MySQL 查询。一些子查询需要很长时间，导致运行时间很长。我已经在表上创建了索引，但仍然需要大约 15 秒才能运行。我需要将运行时间减少到 1 秒。

这是查询：

SELECT p.prd_id, NULL AS manu_name, NULL AS brand_name, NULL AS Categories, NULL AS Item_Color, NULL AS Ink_Type, NULL AS Industry,
 NULL AS Size, NULL AS Decoration_Method, NULL AS Rush_Production, NULL AS Themes, NULL AS Material, NULL AS Pattern, NULL AS Country_Origin, NULL AS Ships_From, pi.prd_img_name
 FROM products p
 LEFT JOIN product_image_p pi ON p.prd_id = pi.prd_id
 WHERE p.is_deleted = 'no' 

 UNION
SELECT p.prd_id, NULL AS manu_name, NULL AS brand_name, NULL AS Categories, NULL AS Item_Color, 
 NULL AS Ink_Type, NULL AS Industry, NULL AS Size, GROUP_CONCAT(decoration.dm_name SEPARATOR ', ') AS Decoration_Method, NULL AS Rush_Production, NULL AS Themes, NULL AS Material, 
 NULL AS Pattern, NULL AS Country_Origin, NULL AS Ships_From, NULL AS prd_img_name
 FROM products p
 JOIN product_dm pdm ON pdm.prd_id = p.prd_id
 JOIN decoration_method decoration ON decoration.dm_id = pdm.dm_id
 WHERE p.is_deleted = 'no'
 group by prd_id

 UNION

SELECT p.prd_id, NULL AS manu_name, NULL AS brand_name, NULL AS Categories, NULL AS Item_Color, 
 NULL AS Ink_Type, NULL AS Industry, NULL AS Size, NULL AS Decoration_Method, GROUP_CONCAT(rush.rush_title SEPARATOR ', ') AS Rush_Production, NULL AS Themes, NULL AS Material, 
 NULL AS Pattern, NULL AS Country_Origin, NULL AS Ships_From, NULL AS prd_img_name
 FROM products p
 JOIN product_rush_title prt ON prt.prd_id = p.prd_id
 JOIN rush_title rush ON rush.rush_id = prt.rush_id
 WHERE p.is_deleted = 'no'
 group by prd_id

 UNION
SELECT p.prd_id, NULL AS manu_name, NULL AS brand_name, NULL AS Categories, GROUP_CONCAT(ao.option_name SEPARATOR ', ') AS Item_Color, 
 NULL AS Ink_Type, NULL AS Industry, NULL AS Size, NULL AS Decoration_Method, NULL AS Rush_Production, NULL AS Themes, NULL AS Material, NULL AS Pattern, 
 NULL AS Country_Origin, NULL AS Ships_From, NULL AS prd_img_name
 FROM products p
 JOIN product_attributes pa ON pa.prd_id = p.prd_id 
 JOIN attributes_options ao ON ao.attr_opt_id = pa.attr_opt_id AND ao.attr_id = 19
 WHERE p.is_deleted = 'no' 
 group by prd_id

 UNION
 SELECT p.prd_id, NULL AS manu_name, NULL AS brand_name, NULL AS Categories, NULL AS Item_Color, 
 GROUP_CONCAT(ao.option_name SEPARATOR ', ') AS Ink_Type, NULL AS Industry, NULL AS Size, NULL AS Decoration_Method, NULL AS Rush_Production, NULL AS Themes, NULL AS Material, 
 NULL AS Pattern, NULL AS Country_Origin, NULL AS Ships_From, NULL AS prd_img_name
 FROM products p
 JOIN product_attributes pa ON pa.prd_id = p.prd_id 
 JOIN attributes_options ao ON ao.attr_opt_id = pa.attr_opt_id AND ao.attr_id = 10
 WHERE p.is_deleted = 'no'
 group by prd_id

 UNION
 SELECT p.prd_id, NULL AS manu_name, NULL AS brand_name, NULL AS Categories, NULL AS Item_Color, 
 NULL AS Ink_Type, GROUP_CONCAT(ao.option_name SEPARATOR ', ') AS Industry, NULL AS Size, NULL AS Decoration_Method, NULL AS Rush_Production, NULL AS Themes, NULL AS Material, 
 NULL AS Pattern, NULL AS Country_Origin, NULL AS Ships_From, NULL AS prd_img_name
 FROM products p
 JOIN product_attributes pa ON pa.prd_id = p.prd_id 
 JOIN attributes_options ao ON ao.attr_opt_id = pa.attr_opt_id AND ao.attr_id = 18
 WHERE p.is_deleted = 'no'
 group by prd_id

 UNION
 SELECT p.prd_id, NULL AS manu_name, NULL AS brand_name, NULL AS Categories, NULL AS Item_Color, 
 NULL AS Ink_Type, NULL AS Industry, GROUP_CONCAT(ao.option_name SEPARATOR ', ') AS Size, NULL AS Decoration_Method, NULL AS Rush_Production, NULL AS Themes, NULL AS Material, 
 NULL AS Pattern, NULL AS Country_Origin, NULL AS Ships_From, NULL AS prd_img_name
 FROM products p
 JOIN product_attributes pa ON pa.prd_id = p.prd_id 
 JOIN attributes_options ao ON ao.attr_opt_id = pa.attr_opt_id AND ao.attr_id IN (1, 13, 14)
 WHERE p.is_deleted = 'no'
 group by prd_id

 UNION
 SELECT p.prd_id, NULL AS manu_name, NULL AS brand_name, NULL AS Categories, NULL AS Item_Color, 
 NULL AS Ink_Type, NULL AS Industry, NULL AS Size, NULL AS Decoration_Method, NULL AS Rush_Production, GROUP_CONCAT(ao.option_name) AS Themes,
 NULL AS Material, NULL AS Pattern, NULL AS Country_Origin, NULL AS Ships_From, NULL AS prd_img_name
 FROM products p
 JOIN product_attributes pa ON pa.prd_id = p.prd_id 
 JOIN attributes_options ao ON ao.attr_opt_id = pa.attr_opt_id AND ao.attr_id = 17
 WHERE p.is_deleted = 'no'
 group by prd_id

 UNION
 SELECT p.prd_id, NULL AS manu_name, NULL AS brand_name, NULL AS Categories, NULL AS Item_Color, 
 NULL AS Ink_Type, NULL AS Industry, NULL AS Size, NULL AS Decoration_Method, NULL AS Rush_Production, NULL AS Themes, GROUP_CONCAT(ao.option_name) AS Material, 
 NULL AS Pattern, NULL AS Country_Origin, NULL AS Ships_From, NULL AS prd_img_name
 FROM products p
 JOIN product_attributes pa ON pa.prd_id = p.prd_id 
 JOIN attributes_options ao ON ao.attr_opt_id = pa.attr_opt_id AND ao.attr_id = 12
 WHERE p.is_deleted = 'no'
 group by prd_id

 UNION
 SELECT p.prd_id, NULL AS manu_name, NULL AS brand_name, NULL AS Categories, NULL AS Item_Color, 
 NULL AS Ink_Type, NULL AS Industry, NULL AS Size, NULL AS Decoration_Method, NULL AS Rush_Production, NULL AS Themes, NULL AS Material, 
 GROUP_CONCAT(ao.option_name) AS Pattern, NULL AS Country_Origin, NULL AS Ships_From, NULL AS prd_img_name
 FROM products p
 JOIN product_attributes pa ON pa.prd_id = p.prd_id 
 JOIN attributes_options ao ON ao.attr_opt_id = pa.attr_opt_id AND ao.attr_id = 2
 WHERE p.is_deleted = 'no'
 group by prd_id

 UNION
 SELECT p.prd_id, NULL AS manu_name, NULL AS brand_name, NULL AS Categories, NULL AS Item_Color, 
 NULL AS Ink_Type, NULL AS Industry, NULL AS Size, NULL AS Decoration_Method, NULL AS Rush_Production, NULL AS Themes, NULL AS Material, 
 NULL AS Pattern, GROUP_CONCAT(ao.option_name) AS Country_Origin, NULL AS Ships_From, NULL AS prd_img_name
 FROM products p
 JOIN product_attributes pa ON pa.prd_id = p.prd_id 
 JOIN attributes_options ao ON ao.attr_opt_id = pa.attr_opt_id AND ao.attr_id = 11
 WHERE p.is_deleted = 'no'
 group by prd_id

 UNION
 SELECT p.prd_id, NULL AS manu_name, NULL AS brand_name, NULL AS Categories, NULL AS Item_Color, 
 NULL AS Ink_Type, NULL AS Industry, NULL AS Size, NULL AS Decoration_Method, NULL AS Rush_Production, NULL AS Themes, NULL AS Material, 
 NULL AS Pattern, NULL AS Country_Origin, GROUP_CONCAT(ao.option_name) AS Ships_From, NULL AS prd_img_name
 FROM products p
 JOIN product_attributes pa ON pa.prd_id = p.prd_id 
 JOIN attributes_options ao ON ao.attr_opt_id = pa.attr_opt_id AND ao.attr_id = 8
 WHERE p.is_deleted = 'no'
 group by prd_id

【问题讨论】：

你检查过哪些是慢的吗？
请向我们解释您的查询
ao.attr_id = 19 的很慢，需要一秒以上。
更新：我删除了所有左连接（并进行了 INNER 连接），除了品牌（我真的需要左连接），时间减少到 5 秒。
又一个 EAV 模式危险的例子。

标签： mysql query-optimization entity-attribute-value

【解决方案1】：

您遇到的问题是重复加入产品属性和产品选项并执行完整查询并为您要查找的每个组件返回一条记录。此外，对于来自选项表的所有部分，您可以为每个产品预查询一次，然后将每个组件作为相关 PRD_ID 的一条记录返回。

为了帮助优化该组件，我强烈建议（如果还没有的话）以下索引...

Table                 Index
product_attributes    ( prd_id, attr_opt_id )
attributes_options    ( attr_opt_id, attr_id, option_name )
products              ( is_deleted, prd_id )
product_dm            ( prd_id, dm_id )
decoration_method     ( dm_id, dm_name )
product_rush_title    ( prd_id, rush_id )
rush_title            ( rush_id, rush_title )

索引将“覆盖”索引，因此系统不必返回原始数据页面来准备大部分结果

然后，将每个部分作为内部预查询执行类似...

由于装饰和抢的内容来自其他表，也可以仅根据产品ID进行预查询和汇总，然后左连接

SELECT
      p2.prd_id,
      GROUP_CONCAT( case when ao.attr_id = 19
                        then ao.option_name else null end SEPARATOR ', ') 
          AS Item_Color,
      GROUP_CONCAT( case when ao.attr_id = 10
                        then ao.option_name else null end SEPARATOR ', ') 
          AS Ink_Type,
      GROUP_CONCAT( case when ao.attr_id = 18
                        then ao.option_name else null end SEPARATOR ', ') 
          AS Industry,
      GROUP_CONCAT( case when ao.attr_id = 17
                        then ao.option_name else null end SEPARATOR ', ') 
          AS Themes,
      GROUP_CONCAT( case when ao.attr_id = 12
                        then ao.option_name else null end SEPARATOR ', ') 
          AS Material,
      GROUP_CONCAT( case when ao.attr_id = 2
                        then ao.option_name else null end SEPARATOR ', ') 
          AS Pattern,
      GROUP_CONCAT( case when ao.attr_id = 11
                        then ao.option_name else null end SEPARATOR ', ') 
          AS Country_Origin,
      GROUP_CONCAT( case when ao.attr_id = 8
                        then ao.option_name else null end SEPARATOR ', ') 
          AS Ships_From,
      GROUP_CONCAT( case when ao.attr_id IN ( 1, 13, 14 )
                        then ao.option_name else null end SEPARATOR ', ') 
          AS Size
   from
      products p2
         JOIN product_attributes pa 
            ON p2.prd_id = pa.prd_id
            JOIN attributes_options ao 
               ON pa.attr_opt_id = ao.attr_opt_id                   
   WHERE 
      p2.is_deleted = 'no'
   group by
      p2.prd_id

然后您可以将整个查询作为子查询滚动到您的主查询中，但使用简化的别名引用和每个前组 concat 字段结果

SELECT 
      p.prd_id, 
      NULL AS manu_name, 
      NULL AS brand_name, 
      NULL AS Categories, 
      preQuery.Item_Color, 
      preQuery.Ink_Type, 
      preQuery.Industry,
      preQuery.Themes, 
      preQuery.Material, 
      preQuery.Pattern, 
      preQuery.Country_Origin, 
      preQuery.Ships_From, 
      preQuery.Size, 
      tmpDeco.Decoration_Method, 
      tmpRush.Rush_Production, 
      pi.prd_img_name
   FROM 
      products p
         LEFT JOIN product_image_p pi 
            ON p.prd_id = pi.prd_id

         LEFT JOIN
         ( THE PRE-QUERY SAMPLE ABOVE ) PreQuery
            ON p.prd_id = PreQuery.prd_id

         LEFT JOIN
         ( select
                 p2.prd_id,
                 GROUP_CONCAT(decoration.dm_name SEPARATOR ', ') AS Decoration_Method 
              FROM 
                 products p2
                    JOIN product_dm pdm 
                       ON p2.prd_id = pdm.prd_id
                       JOIN decoration_method decoration ;
                          ON decoration.dm_id = pdm.dm_id
              WHERE 
                 p2.is_deleted = 'no'
              group by 
                 p2.prd_id ) as tmpDeco
            ON p.prd_id = tmpDeco.prd_id

         LEFT JOIN
         ( select
                 p2.prd_id,
                 GROUP_CONCAT(rush.rush_title SEPARATOR ', ') AS Rush_Production
              FROM 
                 products p2
                    JOIN product_rush_title prt 
                       ON p2.prd_id = prt.prd_id
                       JOIN rush_title rush 
                          ON prt.rush_id = rush.rush_id
              WHERE 
                 p2.is_deleted = 'no'
              group by 
                 p2.prd_id ) as tmpRush
            ON p.prd_id = tmpRush.prd_id

    WHERE
        p.is_deleted = 'no'

希望您能够看到简化的上下文，即一次获取来自同一个表的所有常见描述元素，然后在完成后根据单个别名引用加入。

我可能有一两个类型-o，但认为在其他方面符合您的需求是正确的。您也没有 Manu_Name、Brand_Name、Category 的任何内容，因此您必须完成该组件。

跟进...

根据您的最新情况，我会在您的产品表中添加一些字段来保存每个字段。然后，在为产品添加时为产品属性表创建插入/更新/删除触发器。当有影响时，只需在相关字段上运行简单的 sql-select group_concat() 并立即更新主产品表。是的，这在一定程度上是反规范化的，但完全简化了您执行密集查询和降低性能的需要。

看看我的回答in this other stack question。我正在创建这样一个类似的触发器。这样，您的主表已经预先聚合并准备就绪，并且触发器应该几乎立即运行，因为您只需为单个产品执行单个字段。

【讨论】：

感谢您的努力；但是您建议的查询没有取得任何显着改善。我的查询需要 5.7 秒，而您的查询需要 4.7 秒。我想我还有其他原因导致了瓶颈。
@Mainuddin，为澄清而修改了答案并重定向到另一个实现 TRIGGER 的 Stack 帖子。
我最终决定为搜索创建一个专用表，该表仅包含搜索相关数据。只要产品和其他表发生更改，触发器就会保持更新。这使它像预期的那样超级快。再次感谢@DRapp。

【解决方案2】：

部分性能问题可能是由于“过度标准化”。注意GROUP_CONCAT(ao.option_name) 不是来自属性表，而是来自另一个JOIN。建议你去掉ao，把option_names移到product_attributes。

（这是对@DRapp 答案的补充，从表面上看，它似乎涵盖了查询中的一些弊端。）

【讨论】：