【问题标题】:Priority/Cascading datetime without full table scan没有全表扫描的优先级/级联日期时间
【发布时间】:2017-12-08 05:40:28
【问题描述】:
SELECT IF(priority_date, priority_date, created_at) as created_at
FROM table
WHERE IF(priority_date , priority_date , created_at) 
    BETWEEN '2017-10-10 00:00:00' AND '2017-10-10 23:59:59';

在性能方面执行此查询的最佳方式是什么?

我有一个相当大的表,有两个日期时间。 created_at 和 priority_date。

priority_date 并不总是存在,但如果存在,它应该是被查询的内容,否则它会退回到 created_at。 created_at 总是在创建行时生成。上面的查询会导致(几乎)全表扫描。

初始查询的解释计划:

+------+-------------+-----------------+------+---------------+------+---------+------+--------+-------------+
| id   | select_type | table           | type | possible_keys | key  | key_len | ref  | rows   | Extra       |
+------+-------------+-----------------+------+---------------+------+---------+------+--------+-------------+
|    1 | SIMPLE      | table           | ALL  | NULL          | NULL | NULL    | NULL | 444877 | Using where |
+------+-------------+-----------------+------+---------------+------+---------+------+--------+-------------+

我还应注意priority_date 或created_at 不一定都在同一行的相关时间范围内。所以做类似的事情:

WHERE priority_date BETWEEN '2017-10-10 00:00:00' AND '2017-10-10 23:59:59'
OR created_at BETWEEN '2017-10-10 00:00:00' AND '2017-10-10 23:59:59'

如果 priority_date 是 2017-10-04 23:10:43 并且 created_at 是 2017-10-10 01:23:45 可能会给出不好的结果

该表的当前行:582739

WHERE priority_date BETWEEN... 的计数:3908

WHERE created_at BETWEEN... 的计数:3437

仅在 WHERE BETWEEN 中查询的列之一的示例说明:

+------+-------------+-----------------+-------+----------------------------------+----------------------------------+---------+------+------+-----------------------+
| id   | select_type | table           | type  | possible_keys                    | key                              | key_len | ref  | rows | Extra                 |
+------+-------------+-----------------+-------+----------------------------------+----------------------------------+---------+------+------+-----------------------+
|    1 | SIMPLE      | table           | range | table_created_at_index           | table_created_at_index           | 5       | NULL | 3436 | Using index condition |
+------+-------------+-----------------+-------+----------------------------------+----------------------------------+---------+------+------+-----------------------+

显然IF 不是最有效的。列已编入索引,并且各个行的解释与解释计划中的行计数相匹配。如何在不损失性能的情况下利用优先级/回退查询?

编辑

我能想到的最好的(但是哇,就是那种冗长和复制/粘贴的感觉)

SELECT IF(priority_date, priority_date, created_at) as created_at, priority_date
FROM table 
WHERE priority_date BETWEEN '2017-10-10 00:00:00' AND '2017-10-10 23:59:59'
    OR created_at BETWEEN '2017-10-10 00:00:00' AND '2017-10-10 23:59:59'
HAVING ((priority_date AND priority_date BETWEEN '2017-10-10 00:00:00' AND '2017-10-10 23:59:59')
    OR created_at BETWEEN '2017-10-10 00:00:00' AND '2017-10-10 23:59:59');

及其解释计划:

+------+-------------+-----------------+-------------+-----------------------------------------------------------------------+-----------------------------------------------------------------------+---------+------+------+------------------------------------------------------------------------------------------------------+
| id   | select_type | table           | type        | possible_keys                                                         | key                                                                   | key_len | ref  | rows | Extra                                                                                                |
+------+-------------+-----------------+-------------+-----------------------------------------------------------------------+-----------------------------------------------------------------------+---------+------+------+------------------------------------------------------------------------------------------------------+
|    1 | SIMPLE      | table           | index_merge | table_priority_date_index,table_created_at_index                      | table_priority_date_index,table_created_at_index                      | 6,5     | NULL | 7343 | Using sort_union(table_priority_date_index,table_created_at_index); Using where                      |
+------+-------------+-----------------+-------------+-----------------------------------------------------------------------+-----------------------------------------------------------------------+---------+------+------+------------------------------------------------------------------------------------------------------+

【问题讨论】:

  • 如果“涉及其他列”,则任何答案都可能不适用!

标签: mysql query-optimization


【解决方案1】:

首先你需要一个关于 (priority_date, created_at) 的复合索引,然后你可以使用这样的查询:

SELECT IF(priority_date, priority_date, created_at) as created_at, priority_date
FROM table 
WHERE priority_date BETWEEN '2017-10-10' AND '2017-10-10 23:59:59'
  OR (priority_date IS NULL AND created_at BETWEEN '2017-10-10' AND '2017-10-10 23:59:59');

在复合索引中将 priority_date 放在首位会产生很大的不同。不需要工会。

用 2000 个结果解释 400k 行的结果:

Extra: Using where; Using index
key: priority_created_compound
rows: 2000

【讨论】:

  • 并且添加复合索引后,您可以在priority_date删除单个索引以节省空间。
【解决方案2】:
SELECT priority_date as created_at
FROM table
WHERE priority_date BETWEEN '2017-10-10 00:00:00' AND '2017-10-10 23:59:59'

UNION ALL

SELECT created_at
FROM table
WHERE created_at BETWEEN '2017-10-10 00:00:00' AND '2017-10-10 23:59:59'
 AND priority_date IS NULL;

此查询的前半部分需要一个以priority_date 开头的索引,后半部分需要一个以(created_at, priority_date) 开头的索引。

前半部分自然不会匹配priority_date为NULL的任何行。

后半部分将对created_at做范围条件,然后在匹配行的子集中,进一步测试priority_date是否为NULL。这可以通过索引条件下推来完成。

【讨论】:

  • 两列都已编入索引,因此不会太难满足。您是说第二个联合查询需要复合索引吗?此外,我的示例过于简单,还涉及其他列,因此每个联合都必须列出所有所需的列。
  • 第二个查询将受益于复合索引,是的。阅读 dev.mysql.com/doc/refman/5.6/en/… 了解原因。
【解决方案3】:
( SELECT  priority_date AS created_at
    FROM  table
    WHERE  priority_date >= '2017-10-10'
      AND  priority_date <  '2017-10-10' + INTERVAL 1 DAY )
UNION  DISTINCT 
( SELECT  created_at
    FROM  table
    WHERE  created_at >= '2017-10-10'
      AND  created_at <  '2017-10-10' + INTERVAL 1 DAY
      AND  priority_date IS NULL )

有

INDEX(priority_date, created_at)  -- in this order

注意事项:

  • BETWEEN 的这种方法更适用于 DATETIME 以外的日期范围,并且可以避免计算闰日等。(这没有性能差异。)
  • 对于每个子查询,一个索引是“覆盖”的并且是最优的。不需要 ICP。
  • 我在UNION 上选择了DISTINCT -- 虽然比ALL 慢,但它可能更符合您的应用程序的喜好。如果不能复制,或者复制正常,请切换到ALL。

【讨论】:

    猜你喜欢
    • 1970-01-01
    • 1970-01-01
    • 2016-09-23
    • 1970-01-01
    • 1970-01-01
    • 1970-01-01
    • 1970-01-01
    • 1970-01-01
    • 1970-01-01
    相关资源
    最近更新 更多