如何改进索引以便在毫秒内执行查询？答案

【问题标题】：How to improve indexes in order to get queries executed in milliseconds?如何改进索引以便在毫秒内执行查询？
【发布时间】：2015-04-23 00:09:17
【问题描述】：

我已经失去了整个晚上试图在以下查询中获得更好的性能但没有成功：

    SELECT COUNT(o.id_offer)
      FROM offer o
      JOIN offer_product op
        ON op.id_offer = o.id_offer    
      JOIN advertiser a
        ON a.id_advertiser = o.id_advertiser 
      LEFT 
      JOIN offer_hidden h
        ON h.id_offer = o.id_offer 
       AND h.id_user = 5064
     WHERE o.finality = 'sale' 
       AND h.id_offer IS NULL;
+-------------------+
| COUNT(o.id_offer) |
+-------------------+
|            248250 |
+-------------------+
1 row in set (2.80 sec)

执行大约需要 2 ~ 4 秒。应用程序需要执行大约 8 ~ 10 次类似的查询，因此，毫秒的执行时间是必须的。

解释查询：

+------+-------------+---------------+--------+---------------------------------------------------------+-----------------------------+---------+------------------------------+--------+--------------------------------------+
| id   | select_type | table         | type   | possible_keys                                           | key                         | key_len | ref                          | rows   | Extra                                |
+------+-------------+---------------+--------+---------------------------------------------------------+-----------------------------+---------+------------------------------+--------+--------------------------------------+
|    1 | SIMPLE      | offer         | ref    | PRIMARY,fk_offer_advertiser1_idx,fk_offer_finality1_idx | fk_offer_finality1_idx      | 1       | const                        | 167269 | Using index condition                |
|    1 | SIMPLE      | offer_product | ref    | fk_offer_product_offer1_idx                             | fk_offer_product_offer1_idx | 4       | db.offer.id_offer            |      1 | Using index                          |
|    1 | SIMPLE      | advertiser    | eq_ref | PRIMARY                                                 | PRIMARY                     | 4       | db.offer.id_advertiser       |      1 | Using index                          |
|    1 | SIMPLE      | offer_hidden  | eq_ref | PRIMARY,fk_offer_hidden_user1_idx                       | PRIMARY                     | 8       | db.offer.id_offer,const      |      1 | Using where; Using index; Not exists |
+------+-------------+---------------+--------+---------------------------------------------------------+-----------------------------+---------+------------------------------+--------+--------------------------------------+
4 rows in set (0.00 sec)

和offer 的索引（高基数）：

SHOW INDEXES FROM offer;
+-------+------------+--------------------------+--------------+---------------+-----------+-------------+----------+--------+------+------------+---------+---------------+
| Table | Non_unique | Key_name                 | Seq_in_index | Column_name   | Collation | Cardinality | Sub_part | Packed | Null | Index_type | Comment | Index_comment |
+-------+------------+--------------------------+--------------+---------------+-----------+-------------+----------+--------+------+------------+---------+---------------+
| offer |          0 | PRIMARY                  |            1 | id_offer      | A         |      352683 |     NULL | NULL   |      | BTREE      |         |               |
| offer |          1 | fk_offer_advertiser1_idx |            1 | id_advertiser | A         |      352683 |     NULL | NULL   | YES  | BTREE      |         |               |
| offer |          1 | fk_offer_finality1_idx   |            1 | finality      | A         |           6 |     NULL | NULL   |      | BTREE      |         |               |
+-------+------------+--------------------------+--------------+---------------+-----------+-------------+----------+--------+------+------------+---------+---------------+
3 rows in set (0.04 sec)

我的开发环境目前有大约 30 万个报价和 20 万个广告客户，但生产数据库分别有大约 800 万和 200 万个（分别）。

     SELECT COUNT(*) FROM offer;
+----------+
| COUNT(*) |
+----------+
|   327513 |
+----------+
1 row in set (0.06 sec)


    SELECT COUNT(*) FROM advertiser;
+----------+
| COUNT(*) |
+----------+
|   214885 |
+----------+
1 row in set (0.14 sec)

创建表语句：

CREATE TABLE `offer` (
  `id_offer` int(11) NOT NULL AUTO_INCREMENT,
  `id_advertiser` int(11) unsigned NOT NULL,
  `description` text,
  `date_offer` datetime NOT NULL,
  `finality` enum('buy','sale') NOT NULL,
  PRIMARY KEY (`id_offer`),
  KEY `fk_offer_advertiser1_idx` (`id_advertiser`),
  KEY `fk_offer_finality1_idx` (`finality`),
  CONSTRAINT `fk_offer_advertiser1` FOREIGN KEY (`id_advertiser`) REFERENCES `advertiser` (`id_advertiser`) ON DELETE CASCADE ON UPDATE CASCADE
) ENGINE=InnoDB AUTO_INCREMENT=1 DEFAULT CHARSET=utf8

我错过了什么吗？

编辑 1 - In reply to @Strawberry

CREATE TABLE `advertiser` (
  `id_advertiser` int(11) unsigned NOT NULL AUTO_INCREMENT,
  `name` varchar(200) NOT NULL,
  `first_name` varchar(100) DEFAULT NULL,
  `last_name` varchar(100) DEFAULT NULL,
  `gender` varchar(10) DEFAULT NULL,
  `locale` varchar(10) DEFAULT NULL,
  `created` datetime NOT NULL,
  PRIMARY KEY (`id_advertiser`)
) ENGINE=InnoDB AUTO_INCREMENT=1 DEFAULT CHARSET=utf8

CREATE TABLE `offer_product` (
  `id_offer_product` int(11) NOT NULL AUTO_INCREMENT,
  `id_offer` int(11) NOT NULL,
  `id_product` int(11) NOT NULL,
  PRIMARY KEY (`id_offer_product`),
  KEY `fk_offer_product_offer1_idx` (`id_offer`),
  KEY `fk_offer_product_product1_idx` (`id_product`),
  KEY `fk_offer_product_offer_product1_idx` (`id_offer`,`id_product`),
  KEY `fk_offer_product_product_offer1_idx` (`id_product`,`id_offer`),
  CONSTRAINT `fk_offer_product_offer1` FOREIGN KEY (`id_offer`) REFERENCES `offer` (`id_offer`) ON DELETE CASCADE ON UPDATE CASCADE,
  CONSTRAINT `fk_offer_product_product1` FOREIGN KEY (`id_product`) REFERENCES `product` (`id_product`) ON DELETE NO ACTION ON UPDATE NO ACTION
) ENGINE=InnoDB AUTO_INCREMENT=1 DEFAULT CHARSET=utf8;

 CREATE TABLE `offer_hidden` (
  `id_offer` int(11) NOT NULL,
  `id_user` int(11) NOT NULL,
  PRIMARY KEY (`id_offer`,`id_user`),
  KEY `fk_offer_hidden_user1_idx` (`id_user`),
  CONSTRAINT `fk_offer_hidden_user1` FOREIGN KEY (`id_user`) REFERENCES `user` (`id_user`) ON DELETE CASCADE ON UPDATE CASCADE
) ENGINE=InnoDB DEFAULT CHARSET=utf8

【问题讨论】：

h.id_offer 值是否为 NULL？（至少当 h.id_user = 5064 时。）
@Strawberry，请您解释一下原因。一个offer 总是通过offer.id_advertiser 链接到一个advertiser，即NOT NULL。没有advertiser，你就不能拥有offer。加入是inner。 advertiser 在查询的其他任何地方都没有提到。无论您是否使用advertiser 加入，查询返回的行数都应该相同。我错过了什么？
@Strawberry，如果没有广告客户，您将无法获得报价，因为offer.id_advertiser 是NOT NULL。我错了吗？
@VladimirBaranov 哎呀，你是对的！除非 id_advertiser 0 = '未定义的广告商' - 但我认为这会违反约束
@Strawberry，“除非 id_advertiser 0 = '未定义的广告商'”。定义了一个foreign key。所以，如果offer 有id_advertiser = 0（或任何数字NN），那么advertiser 中必须有一行id_advertiser=0（或任何数字NN）。 inner join 会找到该行。我希望优化器会检测到这样的事情，但显然 MySQL 并不那么聪明。如果您添加过滤器id_advertiser > 0，那会更改查询结果。

标签： mysql sql performance database-performance mariadb

【解决方案1】：

你必须设置多列索引：

类似的东西

ALTER  TABLE offer ADD INDEX(finality, id_offer, id_advertiser);
ALTER  TABLE offer_product ADD INDEX(id_offer);
ALTER  TABLE advertiser ADD INDEX(id_advertiser);
ALTER  TABLE offer_hidden ADD INDEX(id_offer, id_user);

这样，您的搜索将首先按最终确定性检索报价，然后使用每个表的索引进行连接

【讨论】：

【解决方案2】：

-- covering index access to 'offer'
-- (and, perhaps, better caching for 'offer_product' too)
ALTER TABLE offer ADD INDEX(finality, id_offer, id_advertiser);

SELECT COUNT(o.id_offer)
FROM (
    -- do 'left join' first to minimize number of rows
    -- before scanning 'offer_product' and 'advertiser'
    SELECT offer.id_offer, offer.id_advertiser
    FROM offer
    LEFT JOIN offer_hidden h ON
        h.id_offer = offer.id_offer
        AND h.id_user = 5064
    WHERE
        offer.finality = 'sale'
        AND h.id_offer IS NULL
) o
JOIN offer_product op ON
    op.id_offer = o.id_offer
JOIN advertiser a ON
    a.id_advertiser = o.id_advertiser
;

【讨论】：